Terminal device, application server, receiving method, and transmitting method

ABSTRACT

A terminal device (10) includes a transceiver (110), a camera (140), a display (130), and a processor (150). The processor (150) determines a first synchronization signal whose radio quality satisfies a predetermined threshold from a plurality of synchronization signals beamformed and transmitted from the base station (20), and reports the first synchronization signal to the base station (20). The processor (150) superimposes a virtual object corrected by using correction information on a captured image of the camera (140) and displays the image on the display (130). The correction information is information for indicating a position of an area covered by the first synchronization signal with respect to a real object. The correction information includes information regarding a direction of the virtual object to be displayed on the display (130) in the area and a distance from the real object to the area.

FIELD

The present disclosure relates to a terminal device, an applicationserver, a receiving method, and a transmitting method.

BACKGROUND

Services using augmented reality (AR) and virtual reality (VR) areexpected as killer contents for 5th generation mobile communicationsystems (5G New Radio (NR)). For example, in a case of the ARtechnology, a virtual content (hereinafter, also referred to as a“virtual object”) in various forms such as text, icon, or animation canbe superimposed on a real object captured in a real space image andpresented to a user. Non Patent Literature 1 and Non Patent Literature 2disclose use cases and (potential) requirements for services using ARand VR (for example, AR/VR games).

Regarding a technology of superimposing a virtual object on a realobject, Non Patent Literature 3 and Patent Literature 1 disclose twomethods, marker-based recognition and maker-less recognition. In a caseof the marker-based recognition, a relative direction of a camera(imaging unit) with respect to a marker can be estimated according tothe direction or pattern of the marker. In a case where the size of themarker is known, a distance between the marker and the camera (imagingunit) can also be estimated. In a case of the maker-less recognition(natural feature tracking), a relative location and direction withrespect to a target object can be estimated according to prominent pointfeatures (interest point or key point) on the target object.Simultaneous localization and mapping (SLAM) is an example of themaker-less recognition technology. The SLAM is a technology ofperforming self-location estimation and environment map creation inparallel by using an imaging unit such as a camera, various sensors, anencoder, and the like. More specifically, a three-dimensional shape ofan imaged subject is sequentially restored based on a moving imagecaptured by the imaging unit. Then, by associating the restorationresult with a result of detecting the position and posture of theimaging unit, a map of the surrounding environment is created and theposition and posture of the imaging unit in the environment areestimated (recognized).

Furthermore, Non Patent Literature 3 and Patent Literature 1 alsodisclose a technology for improving accuracy in capturing and imagerecognition by combining various sensors (for example, a globalpositioning system (GPS), Wi-Fi, Bluetooth (registered trademark),wireless networking such as mobile networks, a magnetometer (forexample, electronic compass), a gyroscope, and a linear accelerometer)and the like for imaging using a camera.

CITATION LIST Non Patent Literature

Non Patent Literature 1: 3GPP TR 22.842, V17.1.0 (September 2019) 3rdGeneration Partnership Project; Technical Specification Group Servicesand System Aspects; Study on Network Controlled Interactive Services(Release 17)

Non Patent Literature 2: 3GPP TS 22.261 v17.0.1 (October 2019) 3rdGeneration Partnership Project; Technical Specification Group Servicesand System Aspects; Service requirements for next generation newservices and markets (Release 17)

Non Patent Literature 3: Dieter Schmalstieg et al., “AR Textbook”,Mynavi Publishing Corporation, published on Jul. 30, 2018.

Patent Literature

Patent Literature 1: WO 2017/183346 A

SUMMARY Technical Problem

Services using AR/VR are being considered for provision in a large-scalefacility such as a stadium and a concert hall. For example, an AR eventsuch as an AR sport tournament or an AR game tournament may be held atthe stadium, and a spectator may view (watch) the AR event via an ARdevice (a smartphone, an AR head-mounted display (ARHMD), or the like).

In a large-scale facility such as a stadium, it is desirable that userexperiences are commonized in such a way that all spectators view(watch) the same object in real time in order to improve the userexperience by creating a sense of unity among the spectators.

In such a case, the spectators view (watch) the same object (a realobject and a virtual object) from different locations (seats). Since thespectators are at different locations (seats) from each other, even in acase where the same object (the real object and the virtual object) isviewed (watched), the directions (viewing directions) of the objects(the real object and the virtual object) that each spectator canvisually recognize is respectively different. Therefore, in order toprovide an appropriate AR image to the spectators at differentlocations, a technology for appropriately superimposing a virtual objecton an object (real object) in the real world is required. Thistechnology includes capturing, image recognition, and rendering andoutputting/emitting. Among them, the capturing and the image recognitionmay include processing such as alignment, calibration, or tracking.

However, in a large-scale facility (for example, a stadium or a concerthall) assumed as a place where a service using AR is provided, thecapturing and the image recognition using the above-described prior artmay be insufficient.

For example, in a large space such as a stadium, it is assumed that adistance from a spectator stand to a target object (for example, amarker or interest point) that serves as a reference for alignment islong. Further, from the viewpoint of reducing a wearing load of thespectator, it is desirable that a terminal (AR device) is lightweightand compact. In this case, performance of a camera that can be mountedon the AR device (for example, a lens size and a sensor size) and anallowable processing load on the device (for example, processorprocessing capacity or battery capacity) may be limited. Therefore, inan AR device with limited camera performance and allowable processingload, there is a possibility that the capturing and the imagerecognition using a reference object (a target object serving as areference for the capturing and the image recognition, for example, amarker or interest point) for alignment arranged at a location far fromspectator stands cannot be appropriately performed.

Therefore, the present disclosure provides a terminal device, anapplication server, a receiving method, and a transmitting method thatcontribute to improving accuracy in capturing and image recognition whenviewing an AR service using 5G from spectator stands in a large-scalefacility such as a stadium.

It should be noted that the above-mentioned problem or purpose is onlyone of a plurality of problems or purposes that can be solved orachieved by a plurality of embodiments disclosed in the presentspecification.

Solution to Problem

According to the present disclosure, a terminal device is provided. Theterminal device includes a transceiver, a camera for imaging a realobject, a display for displaying an augmented reality image in which avirtual object is superimposed on the real object imaged by the camera,and a processor.

The processor is configured to receive, via the transceiver, at leastone of a plurality of synchronization signals beamformed in directionsdifferent from each other and transmitted from a base station. Theprocessor is configured to determine a first synchronization signalwhose radio quality satisfies a predetermined threshold from the atleast one of the received synchronization signals. The processor isconfigured to transmit a random access preamble by using a random accessoccasion corresponding to the first synchronization signal in order toreport the first synchronization signal to the base station. Theprocessor is configured to receive information regarding the augmentedreality image from an application server after a random accessprocessing procedure including the transmission of the random accesspreamble is completed.

The information regarding the augmented reality image is correctioninformation used for displaying the augmented reality image, oraugmented reality image data in which the virtual object is aligned withrespect to the real object based on the correction information. In acase where the information regarding the augmented reality image is thecorrection information, the processor aligns the virtual object withrespect to the real object by using the correction information,generates the augmented reality image, and outputs the augmented realityimage to the display. In a case where the information regarding theaugmented reality image is the augmented reality image data in which thevirtual object is aligned with respect to the real object based on thecorrection information, the processor outputs the augmented realityimage to the display based on the received augmented reality image data.

The correction information is information for indicating a position ofan area, covered by the beamformed and transmitted first synchronizationsignal, with respect to the real object. The correction informationincludes information regarding a direction of the virtual object to bedisplayed on the display in the area and a distance from the real objectto the area.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for describing an example of information processingaccording to a first embodiment of the present disclosure.

FIG. 2 is a diagram illustrating a conventional transmission pattern ofa synchronization signal block (SSB).

FIG. 3 is a diagram of a rendering server and an AR/VR client related torendering.

FIG. 4 is a diagram illustrating an example of a logical configurationof a communication system according to the first embodiment of thepresent disclosure.

FIG. 5 is a block diagram illustrating an example of a configuration ofa terminal device according to the first embodiment of the presentdisclosure.

FIG. 6 is a block diagram illustrating an example of a configuration ofa base station according to the first embodiment of the presentdisclosure.

FIG. 7 is a block diagram illustrating an example of a configuration ofan application server according to the first embodiment of the presentdisclosure.

FIG. 8 is a sequence diagram illustrating an operation example of thecommunication system according to the first embodiment of the presentdisclosure.

FIG. 9 is a diagram for describing association between a beam and a seatgroup according to the first embodiment of the present disclosure.

FIG. 10 is a diagram for describing the association between the beam andthe seat group according to the first embodiment of the presentdisclosure.

FIG. 11 is a diagram for describing correction information according tothe first embodiment of the present disclosure.

FIG. 12 is a sequence diagram illustrating an operation example of thecommunication system according to a first modified example of the firstembodiment of the present disclosure.

FIG. 13 is a sequence diagram illustrating an operation example of thecommunication system according to a second modified example of the firstembodiment of the present disclosure.

FIG. 14 is a diagram for describing a configuration of a point cloud.

FIG. 15 is a diagram for describing a spatial division method andspatial position information according to a second embodiment of thepresent disclosure.

FIG. 16 is a flowchart for describing generation processing in which anapplication server (media presentation description (MPD) file server)generates a file storing a partial geometry-based point cloudcompression (G-PCC) stream.

FIG. 17 is a flowchart for describing reproduction processing in which aterminal device (MPEG-DASH client) reproduces the file storing thepartial G-PCC stream.

FIG. 18 is a diagram illustrating an example of an applicationarchitecture to which edge computing is applied.

FIG. 19 is a sequence diagram illustrating an example of a processingprocedure of a communication system according to a third embodiment ofthe present disclosure.

FIG. 20 is a sequence diagram illustrating an example of a processingprocedure of the communication system according to the third embodimentof the present disclosure.

FIG. 21 is a sequence diagram illustrating an enhanced cell-ID (E-CID)measurement initiation procedure.

FIG. 22 is a sequence diagram illustrating an E-CID measurement reportprocedure.

FIG. 23 is a sequence diagram illustrating an observed time differenceof arrival (OTDOA) information exchange procedure.

DESCRIPTION OF EMBODIMENTS

Hereinafter, exemplary embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in the present specification and the drawings, components havingsubstantially the same functional configuration are provided with thesame reference signs, so that an overlapping description of thesecomponents is omitted.

In the present specification and the drawings, components havingsubstantially the same functional configuration may be distinguished byadding different alphabets or numerals after the same reference signs.For example, a plurality of components having substantially the samefunctional configuration are distinguished as necessary, such as UEs 10Aand 10B. However, in a case where it is not particularly necessary todistinguish each of the plurality of components having substantially thesame functional configuration, only the same reference sign is given.For example, in a case where it is not necessary to distinguish betweenthe UEs 10A and 10B, it is simply referred to as a UE 10.

Each of the plurality of embodiments (including examples) describedbelow can be implemented independently. On the other hand, at least someof the plurality of embodiments described below may be implemented incombination with at least some of other embodiments as appropriate.These plurality of embodiments may include novel characteristicsdifferent from each other. Therefore, these plurality of embodiments cancontribute to achieve or solving different purposes or problems, and canexert different effects.

Some of the plurality of exemplary embodiments described below aredescribed with 5G New Radio (NR) as a main target. However, theseembodiments are not limited to 5G NR, and may be applied to other mobilecommunication networks or systems such as 3GPP long term evolution (LTE)(including LTE-Advanced and LTE-Advanced Pro), a 3GPP universal mobiletelecommunications system (UMTS), and the like.

The NR is the next generation (5th generation) radio access technology(RAT) following the LTE. The NR is a radio access technology that cansupport various use cases including enhanced mobile broadband (eMBB),massive Internet of Things (mIoT) (or massive machine typecommunications (mMTC)), and ultra-reliable and low latencycommunications (URLLC). The NR has been studied for a technicalframework that addresses usage scenarios, requirements, arrangementscenarios, and the like in those use cases. In addition, the NR includesnew radio access technology (NRAT) and Further EUTRA (FEUTRA).

Note that the description will be provided in the following order.

1.First Embodiment

1.1. Outline of Information Processing According to First Embodiment ofPresent Disclosure

1.2. Overview of Radio Communication between Base Station and UE

1.3. Example of Configuration of Communication System

1.3.1. Example of Overall Configuration of Communication System

1.3.2. Example of Configuration of Terminal Device

1.3.3. Example of Configuration of Base Station

1.3.4. Example of Configuration of Application Server

1.4. Operation of Communication System

1.5. Modified Examples

1.5.1. First Modified Example

1.5.2. Second Modified Example

2. Second Embodiment

3. Third Embodiment

4. Fourth Embodiment

5. Other Embodiments

6. Supplementary Description

1. First Embodiment

<1.1. Outline of Information Processing According to First Embodiment ofPresent Disclosure>

FIG. 1 is a diagram for describing an example of information processingaccording to a first embodiment of the present disclosure. Theinformation processing according to the first embodiment is performed bya communication system including a user equipment (UE) 10 possessed by auser in a spectator stand in a large-scale facility such as a stadiumST, a base station 20 that performs communication with the UE 10, and anapplication server 30 (not illustrated) that generates augmented reality(AR) image data to be presented to a user, but is not limited thereto.

In the information processing according to the first embodiment, thebase station 20 transmits the augmented reality image data generated bythe application server 30 to the UE 10, and processing of displaying theaugmented reality image data on a display of the UE 10 is performed.Hereinafter, an augmented reality image may be referred to as an ARimage. The UE 10 is, for example, AR glasses which are a kind of ARhead-mounted display (ARHMD), and presents the AR image to a user whowears the AR glasses in a spectator stand in the stadium ST. In the ARimage, a virtual object V1 (AR image data) is superimposed on a realobject R1 on the ground in the stadium ST. By viewing the AR image, theuser can watch an AR event held at the stadium ST, such as AR sports andAR game competitions, and participate in the AR event. The real objectR1 can be a moving object such as a ball or a person on the ground, or amarker provided on the ground.

Here, the stadium ST is a large-scale facility, and a plurality of usersview the same virtual object V1 from spectator stands surrounding theground. Therefore, each user views the same object (the real object R1and the virtual object V1) from a different location, but a direction inwhich the object is viewed (viewing direction) is different for eachuser. For example, in the example of FIG. 1, a user possessing a UE 10Aviews the virtual object V1 from the front-left side. That is, a viewingdirection L1 of the user possessing the UE 10A is a direction from thefront-left side of the virtual object V1 toward the virtual object V1.On the other hand, a user possessing a UE 10B views the virtual objectV1 from the front-right side. That is, a viewing direction L2 of theuser possessing the UE 10B is a direction from the front-right side ofthe virtual object V1 toward the virtual object V1.

When the application server 30 generates the same AR image data for eachUE 10, there is a possibility that the UEs 10A and 10B present the ARimages including the virtual object V1 (AR image data) viewed from thesame direction even though the viewing directions L1 and L2 aredifferent, which may give the users a sense of discomfort.

Therefore, in order to generate the AR image data according to theviewing directions L1 and L2 of the users so as not to give the users asense of discomfort, it is necessary to correct the virtual object (ARimage data) based on the viewing direction for each user, that is, eachUE 10, and superimpose the corrected virtual object on the real space.

Here, in conventional methods such as marker-based recognition andmarker-less recognition, the viewing direction of the user is detectedby detecting a marker or an interest point with a camera mounted on ARglasses (corresponding to the UE 10 of the present embodiment). However,in a large-scale facility such as the stadium ST, the size of the markeror interest point may be small, which makes detection using the ARglasses difficult. Furthermore, considering the long-term viewing of theAR image data by the user, the lightness of the AR glasses(corresponding to the UE 10 of the present embodiment) can be one of theimportant factors, and it may be difficult to mount a camera thatenables high-quality imaging.

Therefore, in the information processing according to the presentembodiment, a location to which each of a plurality of beams formed bythe base station 20 is delivered is associated in advance withcorrection information generated for each UE 10 to appropriatelysuperimpose the AR image data generated in a case where the AR imagedata is viewed from the location. As a result, the AR image datacorrected according to the location of the UE 10 (hereinafter, alsoreferred to as corrected AR image data) is presented to the user.

Specifically, in Step S1, a base station 20A transmits synchronizationsignals to the UE 10 while sweeping the beams. For example, in FIG. 1,the base station 20A transmits a plurality of beams B1 to B3 indifferent directions. Note that the number of beams transmitted by thebase station 20A is not limited to three, and may be two or four ormore.

In Step S2, the UE 10A determines a synchronization signal whose radioquality (for example, reception level) satisfies a predeterminedthreshold from the synchronization signals received from the basestation 20A, and determines, as the best beam, a beam that hastransmitted the determined synchronization signal. The UE 10A reportsinformation regarding the determined best beam to the base station 20A.The reported information regarding the best beam is provided to theapplication server 30 via the base station 20A or the like.

The application server 30 generates the corrected AR image datacorresponding to the best beam determined by the UE 10A. The correctedAR image data is associated in advance so that when the user views theAR image data from the viewing direction L1, the virtual object V1 issuperimposed on the real object R1 in an appropriate direction. Forexample, the application server 30 generates the corrected AR image databy correcting the AR image data based on the correction informationcorresponding to the best beam, and transmits the corrected AR imagedata to the UE 10A via the base station 20A.

In Step S3, the UE 10A generates an AR image M1 based on a line-of-sightdirection of the user that corresponds to the best beam by superimposingthe corrected AR image data on a captured image (real object) of acamera mounted on the UE 10A, for example. In the example of FIG. 1, theUE 10A displays, as the AR image M1, an image of the virtual object V1viewed from the viewing direction L1 on the display.

Similarly, the UE 10B determines the best beam from a plurality of beamstransmitted by the base station 20B, and the application server 30generates the corrected AR image data based on the correctioninformation corresponding to the best beam. The UE 10B displays an ARimage M2 (an image of the virtual object V1 viewed from the viewingdirection L2) on the display by superimposing the virtual object V1 ofthe corrected AR image data on the real object, as illustrated in FIG.1.

By associating the beam transmitted by the base station 20 with thecorrection information in this way, it is possible to provide thecorrected AR image data according to the location of the UE 10.Therefore, it is possible to contribute to improving the accuracy incapturing and image recognition when viewing an AR service fromspectator stands in a large-scale facility such as the stadium ST.

Hereinafter, the details of the communication system that performs theabove-described information processing will be described with referenceto the drawings.

<1.2. Overview of Radio Communication Between Base Station and UE>

The UE 10 and the base station 20 described above perform radiocommunication based on, for example, 5G NR. Beamforming performed by 5GNR, especially the base station 20, will be described below.

5G NR allows communication in a high frequency band (for example, a bandof 6 GHz or higher) compared to LTE of the 4th generation cellularcommunication system. In the high frequency band, beamforming is used tocover the characteristics (straightness and attenuation) of radio waves(i.e., to compensate for propagation loss). Thereby, the propagationloss can be compensated by the beam gain. However, beamforming allowsradio waves to travel far, even in the high frequency band, whilenarrowing the beam and narrowing a physical range covered by a singlebeam. Therefore, 3GPP 5G NR introduces beam sweeping. Beamforming is atechnology for sequentially broadcasting a plurality of synchronizationsignals beamformed in different directions from the base station 20 (seeFIG. 4). Therefore, it is possible to cover an area that could becovered without beamforming (i.e., with an omnidirectional beam) in alow frequency band even in the high frequency band. As for a signalsubjected to beam sweeping, at least a synchronization signal(SS/physical broadcast channel (PBCH) block) and a channel stateinformation reference signal (CSI-RS) are specified in a downlinkdirection.

The beam sweeping of the synchronization signal (SS/PBCH block) will bedescribed more specifically. In 3GPP Rel.15, the synchronization signalfor downlink synchronization of the terminal device (UE) 10 with thenetwork is called a synchronization signal block (SSB) (SS/PBCH block).The synchronization signal (SS) includes a primary synchronizationsignal (PSS) and a secondary synchronization signal (SSS). The physicalbroadcast channel (PBCH) carries a master information block. One SSBincludes the PSS, the SSS, and the PBCH. The SSB is periodicallytransmitted from the base station 20 (radio access network (RAN)) into acell as an SSB burst (SS burst) including a plurality of SSBs. An SSBindex as an identifier is added to each of the plurality of SSBs in oneSSB burst. In 3GPP Rel.15, the number of SSBs in one SSB burst is either4, 8, or 64 according to a frequency range. The SSB is beamformed andtransmitted in different directions. The terminal device 10 reports, tothe base station 20, a beam of a direction whose reception quality isfavorable in a random access channel (RACH) occasion associated with theSSB index.

The frequency band and the number of beams (the number of SSBs) per unittime (e.g., one SS burst or one SSB burst) are defined in associationwith each other. In 3GPP, the maximum number of beams (the number ofSSBs) per unit time (e.g., one SS burst or one SSB burst) is defined asLmax. For example, a band with a carrier frequency of 6 GHz or lesscorresponds to a frequency range FR 1. A band with a carrier frequencyof 6 GHz or higher corresponds to a frequency range FR2.

FIG. 2 is a diagram illustrating a transmission pattern of aconventional SSB. Cases A to E are transmission patterns of theconventional SSB. For FR1 (i.e., Cases A to C), the number oftransmitted SSBs is four or eight per unit time (half frame: 5 ms or oneSSB burst). For FR2 (i.e., Cases D and F), the number of transmittedSSBs is 64 per unit time (half frame: 5 ms or one SSB burst). In otherwords, since FR2 is a frequency range of 24250 MHz to 52600 MHz, Lmax=64is defined as the number of SSBs that can be supported even in thisfrequency band.

In other words, in a case of FR2 (a band of 6 GHz or higher), a maximumof 64 (64 types of) beamformed SSBs are required, which is more thanthat in a case of FR1. In other words, in a case of FR1 (Cases A to C),the maximum number of SSBs transmitted per unit time (half frame: 5 ms)is four or eight. Therefore, in a case of FR1, it is sufficient toperform beam sweeping with a maximum of four or eight beams (beamformedSSBs). However, in a case of FR2 (Cases D and F), the maximum number ofSSBs transmitted per unit time (half frame: 5 ms) is 64. Therefore, in acase of FR2, it is necessary to perform beam sweeping with a maximum of64 beams (beamformed SSBs). This is because in the high frequency band(for example, a band of 6 GHz or higher), the propagation loss becomeslarger than that in the low frequency band, and it is necessary tonarrow down the beam.

In the future, a frequency band higher than 52600 MHz (for example, aband of 100 GHz) and a frequency range (for example, FR3) may be newlydefined by expansion. In this case, 64 may not be enough for the maximumnumber of SSBs (Lmax) in one SSB burst to cover the same geographicalarea because it is necessary to further narrow the beam. For example, inthe band of 100 GHz, Lmax=64 is not sufficient, and Lmax may be largerthan 64, for example 128 or 256. Some embodiments, including the presentembodiment, are also applicable to a frequency range (e.g., FR3) andLmax of 64 or more that may be defined in the future.

As can be understood from the characteristics of the synchronizationsignal (SSB) in 5G NR described above, the SSB (i.e., beam) preferablefor the terminal device (UE) 10 (i.e., radio quality is equal to orhigher than the predetermined threshold) varies depending on thelocation of the terminal device (UE) 10. Which SSB (i.e., beam) ispreferable for the terminal device (UE) 10 can be determined based on 5ms+several ms (e.g., one SS burst+processing time in the terminal).Therefore, in the present embodiment, the SSB index is associated withthe correction information (direction and distance) applied to thevirtual object in the AR/VR image.

As described above, 3GPP TR 22.842 v17.1.0 and TS 22.261 v17.0.1 specifythe requirements for rendering a game image for a cloud game usingAR/VR. More specifically, these technical specifications and reportsdescribe motion-to-photon latency and motion-to-sound latency asallowable latencies at a level that allows an AR/VR user to feelcomfortable with a motion in a video in rendering a game image, asfollows.

Motion-to-photon latency: The motion-to-photon latency is in a range of7 to 15 ms while maintaining a required data rate (1 Gbps).

Motion-to-sound latency: less than 20 ms.

Note that the motion-to-photon latency can be defined as latency betweena physical motion of the user's head and an updated image in an AR/VRheadset (e.g., head-mounted display). Also, the motion-to-sound latencycan be defined as latency between the physical motion of the user's headand updated sound waves that reach the user's ears from a head-mountedspeaker. The AR/VR headset (head-mounted display) and the head-mountedspeaker here may be the terminal device 10 in the present embodiment.

The above technical specifications and reports specify that a 5G systemneeds to satisfy the following two requirements for rendering in orderto satisfy these latency conditions.

Max Allowed End-to-end latency: 5 ms (that is, (e.g., a total allowablelatency in uplink and downlink between the terminal device (UE) 10 inthe present embodiment and an interface for a data network (e.g., anetwork deployed ahead of a core network when viewed from the UE,including a cloud network or edge network) is 5 ms).

Service bit rate: user-experienced data rate: 0.1 Gbps (100 Mbps) (thatis, a throughput that can support an AR/VR content).

Note that the rendering here includes cloud rendering, edge rendering,or split rendering. In the cloud rendering, AR/VR data is rendered on acloud of the network (on an entity that is based on core network(including the user plane function (UPF)) deployment that does notconsider the location of the user and data network (including theapplication server and application function (AF)) deployment). In theedge rendering, AR/VR data is rendered on an edge of the network (on anentity (e.g. an edge computing server (the application server 30 in thedata network in network deployment for edge computing) that is based oncore network (including the UPF) deployment and data network (includingthe application server and AF) deployment close to the location of theuser). The split rendering means rendering in which a part of therendering is performed on the cloud and the other part is performed onthe edge.

FIG. 3 is a diagram illustrating images of a rendering server and anAR/VR client related to rendering. Note that the rendering server andthe AR/VR client are described in the above technical report. Here, theAR/VR client may correspond to the terminal device (UE) 10 in thepresent embodiment. Further, a cloud render server may be an applicationserver arranged on the cloud, or an application server (e.g., edgecomputing server) arranged on the edge for edge computing. Further, thecloud render server may be referred to as an edge render server or asplit render server. The rendering server may correspond to theapplication server 30 in the present embodiment.

<1.3. Example of Configuration of Communication System>

<1.3.1. Example of Overall Configuration of Communication System>

FIG. 4 is a diagram illustrating an example of a logical configurationof the communication system according to the first embodiment of thepresent disclosure. The communication system of FIG. 4 includes theterminal device (UE) 10, the base station (gNB) 20, a core network node(e.g., UPF) 40, and the application server (e.g., (edge) applicationserver) 30.

(Terminal Device)

The terminal device 10 may be connected to the base station 20 via a Uuinterface. More specifically, the terminal device (UE) 10 performs acell search/cell selection procedure, camps on a certain cell as asuitable cell, and then performs a random access procedure at anarbitrary timing. From the viewpoint of the terminal, the random accessprocedure includes transmission of a random access preamble, receptionof a random access response, and subsequent reception of Message 3(Msg3). After the random access procedure succeeds, a radio resourcecontrol (RRC) setup procedure is performed with the base station (gNB)20, and the terminal device 10 enters RRC Connected in response toreception of an RRC setup message. Then, the terminal device 10considers a current cell (serving cell) in which the RRC setup procedureis performed as a primary cell (PCell).

(Base Station)

As described above, the base station 20 performs communication with theterminal device 10 via the Uu interface. Note that the single basestation 20 may manage a plurality of cells or a plurality of BWPs. Oneor more base stations 20 constitute a radio access network (RAN). Here,the radio access network may be an evolved universal terrestrial radioaccess network (E-UTRAN) or a next generation radio access network(NG-RAN). Further, the base station 20 may be referred to as any one ora combination of a gNB central unit (CU) and a gNB distributed unit(DU). In the present embodiment, the base station 20 may be configuredto be capable of performing radio communication with another basestation. For example, in a case where a plurality of base stations 20are eNBs or a combination of eNB(s) and gNB(s), the devices may beconnected by an X2 interface. Further, in a case where a plurality ofbase stations 20 are eNBs or a combination of eNB(s) and gNB(s), thedevices may be connected by an Xn interface. Further, in a case where aplurality of base stations 20 are a combination of gNB CU(s) and gNBDU(s), the devices may be connected by an F1 interface. All or at leastsome of the messages/information to be described later may becommunicated between a plurality of base stations 20 (for example, viathe X2, Xn, or F1 interface).

Further, the base station 20 may include a set of a plurality ofphysical or logical devices. For example, in the present embodiment, thebase station 20 is classified into a plurality of devices including abaseband unit (BBU) and a radio unit (RU), and may be interpreted as aset of these plurality of devices. In addition or instead, in theembodiments of the present disclosure, the base station 20 may be eitheror both of the BBU and the RU. The BBU and the RU may be connected by apredetermined interface (for example, eCPRI). In addition or instead,the RU may be referred to as a remote radio unit (RRU) or a Radio DoT(RD). In addition or instead, the RU may correspond to the gNB DUdescribed above or below. In addition or instead, the BBU may correspondto the gNB CU described above or below. In addition or instead, the RUmay be a device integrally formed with an antenna. An antenna of thebase station 20 (for example, the antenna integrally formed with the RU)may adopt an advanced antenna system and support MIMO (for example,FD-MIMO) or beamforming. In the advanced antenna system, the antenna ofthe base station 20 (for example, the antenna integrally formed with theRU) may include, for example, 64 transmission antenna ports and 64reception antenna ports. In a case where the base station 20 supportsbeamforming, the base station 20 transmits a signal by, for example,performing beam sweeping of the beam in a circumferential direction or aradial direction of a cell, as illustrated in FIG. 4. Note that thedirection of the beam sweeping is not limited to a horizontal direction,and may be a vertical direction or an arbitrary direction correspondingto a combination of the horizontal direction and the vertical direction.That is, in a case where a plurality of antenna elements of an antennathat performs beamforming are arranged in the horizontal direction andthe vertical direction with respect to an antenna surface, configurationrelated to the antenna to be described later (e.g., an antenna tiltangle, a distance/wavelength between the antenna elements, a phaseoffset, and reference transmit power) can be adjusted to perform adirectivity control of the beam in the horizontal direction and thevertical direction.

(Core Network Node)

The core network node 40 is connected to the base station 20 via anetwork interface. The core network is formed by a plurality of corenetwork nodes 40. The core network may be 5GC. That is, the core networknode 40 may be any one of an access and mobility management function(AMF), a UPF, a session management function (SMF), a network exposurefunction (NEF), an AF, and the like. In FIG. 4, only one core networknode 40 is illustrated, but the number of core network nodes 40 is notlimited thereto. The number of core network nodes 40 that can performcommunication with the base station 20 (e.g., gNB) (i.e., having areference point with the base station (gNB) 20) may be plural.Similarly, the number of core network nodes 40 that can performcommunication with the application server 30 (i.e., having a referencepoint with the application server 30) may be plural. For example, in acase where the core network node 40 is a UPF as illustrated in FIG. 4,the UPF is connected to the base station gNB via an NG-U interface. Inthe NG-U interface, an NG-application protocol (NG-AP) message can becommunicated. All or at least some of the messages/information to bedescribed later may be communicated between the base station 20 and thecore network node 40 (for example, via the NG-C interface or the NG-Uinterface). Also, from the viewpoint of a control plane, the corenetwork node (e.g., AMF) 40 can perform NAS signaling with the terminaldevice 10. That is, all or at least some of the messages/information tobe described later may be communicated between the terminal device 10and the core network node 40 by NAS signaling. As will be describedlater, in a case where the application server 30 in the presentembodiment is an edge application server in an edge data network, thecore network node 40 may be a local UPF.

(Application Server)

The application server ((edge) application server) 30 hosts anapplication provided to the terminal device 10 and data thereof, andprovides application data (e.g., AR image data) in response to a requestfrom the terminal device 10. The application data is provided to theterminal device 10 via the core network and the base station 20.

In a case where the core network node 40 described above is a node(e.g., UPF) in charge of the user plane function, the application server30 is directly or indirectly connected to the core network node 40. Morespecifically, the UPF is operated as a gateway for the data network,communication with a server (e.g., application server 30) within thedata network is enabled. In a case where the core network node 40 is anode (e.g., AMF or SMF) in charge of the control plane function, theapplication server 30 is directly or indirectly connected to the corenetwork node 40. More specifically, the application server (e.g.,application function in a server) 30 can perform communication (e.g.,information exchange using an application programming interface (API) orthe like) with a C-plane node of 5GC directly or indirectly via thenetwork exposure function (NEF).

Note that the edge computing may be applied to the present embodiment.The edge computing allows services of an operator and a third party tobe hosted near an access point of the UE 10. Therefore, end-to-endlatency and a load on a transport network can be reduced, and efficientservice delivery can be realized. That is, the data network may be anedge data network. The application server 30 may be an edge applicationserver in the edge data network. The edge computing here may be referredto as multi-access edge computing (MEC) or mobile edge computing (MEC).Details of an example of application of the present embodiment to theedge computing will be described later in a third embodiment.

<1.3.2. Example of Configuration of Terminal Device>

Next, an example of a configuration of the terminal device 10 accordingto the first embodiment of the present disclosure will be described withreference to FIG. 5. FIG. 5 is a block diagram illustrating an exampleof the configuration of the terminal device 10 according to the firstembodiment of the present disclosure.

For example, the terminal device 10 can be a head-mounted device (e.g.,eyeglasses or goggles), that is, an HMD. For example, the terminaldevice 10 may adopt various structures such as a glass type and a helmettype. The terminal device 10 for displaying an AR image is classifiedinto a video see-through type head-mounted display (HMD) or an opticalsee-through type HMD. Further, the terminal device 10 may be a contactlens type display. The HMD and the contact lens type display aresometimes collectively referred to as a near eye display. Note that theterminal device 10 may be a see-closed HMD compatible with VR, but isnot limited thereto. For example, the terminal device 10 may be aretinal projection type HMD. Alternatively, the terminal device 10 maybe a smartphone, or may be an information processing device including animaging unit (e.g., camera) and a display unit (e.g., display), otherthan the smartphone.

As illustrated in FIG. 5, the terminal device 10 includes an antennaunit 100, a communication unit (transceiver) 110, a storage unit(memory) 120, a display unit (display) 130, an imaging unit (camera)140, and a control unit (processor) 150. Note that the configurationillustrated in FIG. 5 is a functional configuration, and a hardwareconfiguration may be different from this. Further, the functions of theterminal device 10 may be distributed to and implemented in a pluralityof physically separated components.

The antenna unit 100 radiates a signal output from the communicationunit 110 into a space, as radio waves. Further, the antenna unit 100converts radio waves in the space into a signal and outputs the signalto the communication unit 110.

The communication unit 110 transmits and receives a signal. For example,the communication unit 110 receives a downlink signal from the basestation 20 and transmits an uplink signal to the base station 20.

The storage unit 120 is a storage device, from which data can be readand in which data can be written, such as a DRAM, an SRAM, a flashmemory, or a hard disk. The storage unit 120 functions as a storagemeans of the terminal device 10.

The display unit 130 is a display that displays the AR image datatransmitted from the application server 30. The display unit 130 may bean optical see-through display or a non-transmissive (see-closed)display, that is, a video see-through display. In a case where thedisplay unit 130 is the optical see-through display, the display unit130 has optical transparency and displays a virtual object included inthe AR image data on the display under the control of the control unit150. In a case where the display unit 130 is the video see-throughdisplay, the display unit 130 superimposes and displays a virtual objectincluded in the AR image data on a real image captured by the imagingunit 140 under the control of the control unit 150.

The imaging unit 140 is a camera that images the line-of-sight directionof the user. The imaging unit 140 captures an image in front of theuser. As described above, in a case where the display unit 130 is thevideo see-through display, the images in front of the user captured bythe imaging unit 140 may be sequentially displayed on the display unit130.

The control unit 150 is a controller that controls each unit of theterminal device 10. The control unit 150 is implemented by, for example,a processor (hardware processor) such as a central processing unit (CPU)or a microprocessing unit (MPU). For example, the control unit 150 isimplemented in a manner in which the processor executes various programsstored in the storage device inside the terminal device 10 by using arandom access memory (RAM) or the like as a work area. Note that thecontrol unit 150 may be implemented by an integrated circuit such as anapplication-specific integrated circuit (ASIC) or a field programmablegate array (FPGA). The CPU, the MPU, the ASIC, and the FPGA can all beregarded as the controller.

Note that, in addition to the above-described components, the terminaldevice 10 may have a component such as an input/output unit or an audiooutput unit such as a speaker.

<1.3.3. Example of Configuration of Base Station>

Next, an example of a configuration of the base station 20 according tothe first embodiment of the present disclosure will be described withreference to FIG. 6. FIG. 6 is a block diagram illustrating an exampleof the configuration of the base station 20 according to the firstembodiment of the present disclosure.

As illustrated in FIG. 6, the base station 20 includes an antenna unit200, a communication unit (transceiver) 210, a network communicationunit (NW interface) 220, a storage unit (memory) 230, and a control unit(processor) 240. Note that the configuration illustrated in FIG. 6 is afunctional configuration, and a hardware configuration may be differentfrom this. Further, the functions of the base station 20 may bedistributed to and implemented in a plurality of physically separatedcomponents.

The antenna unit 200 radiates a signal output from the communicationunit 210 into a space, as radio waves. Further, the antenna unit 200converts radio waves in the space into a signal and outputs the signalto the communication unit 210.

The communication unit 210 transmits and receives a signal. For example,the communication unit 210 receives an uplink signal from the terminaldevice 10 and transmits a downlink signal to the terminal device 10.

The network communication unit 220 is a communication interface forperforming communication with a node located higher on the network (forexample, the core network node 40 (see FIG. 4)). For example, thenetwork communication unit 220 is a LAN interface such as an NIC.Further, the network communication unit 220 may be a wired interface ora wireless interface. The network communication unit 220 functions as anetwork communication means of the base station 20.

The storage unit 230 is a storage device, from which data can be readand in which data can be written, such as a DRAM, an SRAM, a flashmemory, or a hard disk. The storage unit 230 functions as a storagemeans of the base station 20.

The control unit 240 is a controller that controls each unit of the basestation 20. The control unit 240 is implemented by, for example, aprocessor (hardware processor) such as a central processing unit (CPU)or a microprocessing unit (MPU). For example, the control unit 240 isimplemented in a manner in which the processor executes various programsstored in the storage device inside the base station 20 by using arandom access memory (RAM) or the like as a work area. Note that thecontrol unit 240 may be implemented by an integrated circuit such as anapplication specific integrated circuit (ASIC) or a field programmablegate array (FPGA). The CPU, the MPU, the ASIC, and the FPGA can all beregarded as the controller.

<1.3.4. Example of Configuration of Application Server>

Next, an example of a configuration of the application server 30according to the first embodiment of the present disclosure will bedescribed with reference to FIG. 7. FIG. 7 is a block diagramillustrating an example of the configuration of the application server30 according to the first embodiment of the present disclosure.

As illustrated in FIG. 7, the application server 30 includes a networkcommunication unit (NW interface) 310, a storage unit (memory) 320, anda control unit (processor) 330. Note that the configuration illustratedin FIG. 7 is a functional configuration, and a hardware configurationmay be different from this. Further, the functions of the applicationserver 30 may be distributed to and implemented in a plurality ofphysically separated components.

The network communication unit 310 is a communication interface forperforming communication with a node located on the network (forexample, the core network node 40 (see FIG. 4)). For example, thenetwork communication unit 310 is a LAN interface such as an NIC.Further, the network communication unit 310 may be a wired interface ora wireless interface. The network communication unit 310 functions as anetwork communication means of the application server 30.

The storage unit 230 is a storage device, from which data can be readand in which data can be written, such as a DRAM, an SRAM, a flashmemory, or a hard disk. The storage unit 230 stores, for example, a beamtransmitted by the base station 20 (for example, the SSB index) and thecorrection information of the AR image data in association with eachother. The storage unit 230 functions as a storage means of theapplication server 30.

The control unit 240 is a controller that controls each unit of theapplication server 30. The control unit 240 is implemented by, forexample, a processor (hardware processor) such as a central processingunit (CPU) or a microprocessing unit (MPU). For example, the controlunit 240 is implemented in a manner in which the processor executesvarious programs stored in the storage device inside the applicationserver 30 by using a random access memory (RAM) or the like as a workarea. Note that the control unit 240 may be implemented by an integratedcircuit such as an application specific integrated circuit (ASIC) or afield programmable gate array (FPGA). The CPU, the MPU, the ASIC, andthe FPGA can all be regarded as the controller.

<1.4. Operation of Communication System>

FIG. 8 is a sequence diagram illustrating an operation example of thecommunication system according to the first embodiment of the presentdisclosure. In this operation example, scenes (situations) that operatein Steps S101 and S102, which are preparatory steps, and scenes thatoperate in subsequent Steps S103 to S108 can be distinguished. That is,this operation example includes, for example, the preparatory steps(Steps S101 and S102) performed before the AR/VR event is held, andSteps S103 to S108 repeatedly performed during the AR/VR event.

In Step S101, as an advance preparation, a transmission direction ofeach beam transmitted from the base station 20 (gNB) is adjusted. Forexample, the configuration related to the antenna of the base station 20(gNB) (e.g., the antenna tilt angle, the distance/wavelength between theantenna elements, the phase offset, and the reference transmit power)can be adjusted so that a predetermined area can be covered by aplurality of beams.

For example, in a two-dimensional direction, a direction of the beam atan arbitrary angle θ can be obtained from the following equation.

$\begin{matrix}\left\lbrack {{Math}1} \right\rbrack &  \\{{\Delta\varphi} = \frac{2\pi d\sin\theta}{\lambda}} & \left( {{Equation}1} \right)\end{matrix}$

Here, d is the distance between the plurality of antenna elements, λ isthe wavelength of the signal, and Δφ is the phase offset. For example,the configuration related to the antenna is adjusted by using (applying)(Equation 1) so that a predetermined area can be covered by a pluralityof beams. In a case where the use case to which the present embodimentis applied is an AR/VR event at a large-scale facility such as thestadium ST, an area covered by one beam may be associated with a seatgroup including one or more seats. FIGS. 9 and 10 illustrate an exampleof a case of associating a plurality of beams (SSBs) with a seat groupincluding one or more seats in a large-scale facility such as thestadium ST. FIGS. 9 and 10 are diagrams for describing the associationbetween the beam and the seat group according to the first embodiment ofthe present disclosure.

For example, as described above, in a case of FR2 (a band of 6 GHz orhigher), the maximum number of SSBs transmitted per unit time (halfframe: 5 ms) is 64. In other words, beams are formed in 64 differentdirections, and beams in 64 directions are sequentially transmitted(broadcast) from the base station 20 (gNB) during a unit time (one SSburst). Therefore, the configuration of the antenna of the base station20 (gNB) is adjusted so that one beam corresponds to a seat groupincluding one or more seats. In other words, in a case where theterminal device 10 is located in a seat group including one or moreseats, the antenna configuration is adjusted so that a predeterminedbeam (SSB) becomes the best beam for the terminal device 10 (the bestbeam whose radio quality is the best). As a result, it is possible toassociate the spectator seat(s) with beams (i.e., 64 SSB indexes) in 64different directions.

Next, in Step S102, information (correction information) regarding arelative position of an area that can be covered by one beam withrespect to a reference point for displaying the AR image, and acorresponding beam are associated per each of the plurality of beams.

More specifically, a position (e.g., latitude/longitude) of thereference point (real object) on which the virtual object is to besuperimposed is set in advance. Then, the correction information (thedirection and distance of an area covered by one beam from the referencepoint) is set so that the virtual object is superimposed on thereference point in an appropriate direction and distance when the useradjusts the camera (e.g., the camera 140 provided in the terminal device10 (see FIG. 5)) to be oriented toward the reference point from an areacovered by a certain beam (SSB), and the correction information and acorresponding beam (SSB) are associated with each other. The referencepoint may be simple enough that the presence of the reference point canbe recognized by the camera 140 included in the terminal device 10. Forexample, it does not have to be complicated (highly accurate) enough toidentify the direction or pattern of a marker required for themarker-based recognition. Similarly, there may be no prominent pointfeature (interest point or key point) on a target object required forthe marker-less recognition (i.e., the camera does not have to be ahigh-precision camera enough to enable recognition). This is because thedirection and distance with respect to the reference point can beidentified by the correction information (the direction and distance ofthe area covered by one beam from the reference point).

A specific example of the correction information will be described. FIG.11 is a diagram for describing the correction information according tothe first embodiment of the present disclosure. In FIG. 11, it isassumed that the AR image is viewed from a predetermined spectatorseat(s) in a large-scale facility such as the stadium ST. A case wherethe correction information (the direction and distance of an areacovered by one beam from the reference point) is set so that the virtualobject is superimposed on the reference point in an appropriatedirection and distance in such a case will be described.

FIG. 11 includes an x-y plane when a large-scale facility (for example,the stadium ST) is viewed from directly above (in a directionperpendicular to the ground on the earth) and a z-(x-y) planeperpendicular to the x-y plane. In the x-y plane of FIG. 11, a distancebetween a central portion of an area covered by a beam identified by anSSB index #23 (an area where radio quality of the beam identified by theSSB index #23 is best for the terminal device 10 when the terminaldevice 10 measures the synchronization signal) and the reference point(e.g., a point serving as a reference for superimposition of the ARimage data (virtual object) such as a central point in a large-scalefacility) is L, and an angle from an x-axis direction of the x-y planeis α.

Furthermore, in the z-(x-y) plane of FIG. 11, a distance between thecentral portion of the area covered by the beam identified by the SSBindex #23 and the reference point is represented by L′, and an anglebetween L and L′ in the z-(x-y) plane is β. Note that, in the z-(x-y)plane, the reference point is provided at a point having a height h fromthe ground, but the height h may be zero. Here, in a case where x, y,and z coordinates of the central portion of the area covered by the beamidentified by the SSB index #23 when x, y, and z coordinates of thereference point are (0, 0, 0) are (X_(SSB_23), Y_(SSB_23), Z_(SSB_23)),Equation 2 is valid.

$\begin{matrix}\left\lbrack {{Math}2} \right\rbrack &  \\{\begin{pmatrix}X_{{{SSB}\_}23} \\Y_{{{SSB}\_}23} \\Z_{{{SSB}\_}23}\end{pmatrix} = \begin{pmatrix}{{L}^{\prime}\cos\alpha\cos\beta} \\{{L}^{\prime}\sin{\alpha cos}\beta} \\{{L^{\prime}\sin\beta} + h}\end{pmatrix}} & \left( {{Equation}2} \right)\end{matrix}$

Therefore, (L′, α, and β) obtained from Equation 3 which is amodification of Equation 2 are associated, as the correction informationused for aligning the AR image to be displayed on the display 130 of theterminal device 10 for which radio quality of the beam of the SSB index#23 is best, with the corresponding beam (i.e., SSB index).

$\begin{matrix}\left\lbrack {{Math}3} \right\rbrack &  \\{\begin{pmatrix}L^{\prime} \\\alpha \\\beta\end{pmatrix} = \begin{pmatrix}\sqrt{{X_{{{SSB}\_}23}}^{2} + {Y_{{{SSB}\_}23}}^{2} + \left( {Z_{{{SSB}\_}23} - h} \right)^{2}} \\{\tan^{- 1}\frac{Y_{{{SSB}\_}23}}{X_{{{SSB}\_}23}}} \\{\cos^{- 1}\frac{\sqrt{{X_{{{SSB}\_}23}}^{2} + {Y_{{{SSB}\_}23}}^{2}}}{\sqrt{{X_{{{SSB}\_}23}}^{2} + {Y_{{{SSB}\_}23}}^{2} + \left( {Z_{{{SSB}\_}23} - h} \right)^{2}}}}\end{pmatrix}} & \left( {{Equation}3} \right)\end{matrix}$

These associations may be made prior to service provision to theterminal device 10. For example, the application server 30 may acquireinformation regarding the beam (e.g., the SSB index and a correspondingantenna configuration information list) from the base station 20 (gNB)via the core network (e.g., the core network node 40). For example, theapplication server 30 may acquire the information regarding the beam viathe base station 20 (gNB) or the API provided by the core network. In acase where the present embodiment is applied to the edge computing, theAPI is provided to the application server 30 via a reference point“EDGE-7” between the 3GPP Network (including the base station 20 (gNB)and the core network) and the edge application server. For example, theedge application server can access a 3GPP network function and the API(via the API exposed by the NEF). In addition or instead, the API may beprovided to the application server 30 via a reference point “EDGE-2”between the 3GPP Network (including the base station 20 (gNB) and thecore network) and an edge enabler server, and a reference point “EDGE-3”between the edge enabler server and the edge application server. Inaddition or instead, the API may be provided to the application server30 via a reference point “EDGE-8” between the 3GPP Network (includingthe base station 20 (gNB) and the core network) and an edge data networkconfiguration server, a reference point “EDGE-6” between the edge datanetwork configuration server and the edge enabler server, and thereference point “EDGE-3” between the edge enabler server and the edgeapplication server. Details thereof will be described later.

Return to the description of FIG. 8. In Step S103, the base station 20(gNB) broadcasts the synchronization signal (e.g., SSB) in the cell.More specifically, the base station 20 (gNB) broadcasts a plurality ofsynchronization signals in different directions by performing beamsweeping in which a plurality of synchronization signals are beamformedin different directions and sequentially transmitted. For eachsynchronization signal transmitted in this step, the angle and referencetransmit power are adjusted in advance in the configuration related tothe antenna in Step S101. For example, in the present embodiment, beamsare formed in 64 different directions, and beams in 64 directions aresequentially transmitted (broadcast) from the base station 20 (gNB)during a unit time (one SS burst).

In Step S104, the terminal device 10 receives (detects) at least onesynchronization signal (e.g., SSB) transmitted by beam sweeping andmeasures radio quality of each synchronization signal. The radio qualityhere may be, but is not limited to, any one or a combination ofreference signal received power (RSRP), reference signal receivedquality (RSRQ), signal interference to noise ratio (SINR), receivedsignal strength indicator (RSSI), and channel state information (CSI).The RSRP here may be secondary synchronization signal reference signalreceived power (SS-RSRP). The RSRQ here may be secondary synchronizationsignal reference signal received quality (SS-RSRQ). The SINR here may besecondary synchronization signal interference to noise ratio (SS-SINR).That is, a measurement target of the synchronization signal (e.g., SSB)may be limited to the secondary synchronization signal. Then, in a casewhere the radio quality of the measured synchronization signal is higherthan a predetermined threshold, the synchronization signal is determinedas a synchronization signal of the best beam for the terminal device 10.The best beam for the terminal device 10 (beam A in FIG. 8) is reportedto the network by using a synchronization signal index (e.g., SSBindex). In a case where there are a plurality of synchronization signalswhose radio quality is higher than the predetermined threshold, theterminal device 10 may report only the beam having the highest radioquality to the network, or report a plurality or all of the beamssatisfying the predetermined threshold to the network. The report ismade in Step S105.

In Step S105, the terminal device 10 performs the random accessprocedure with the base station 20 (gNB). As described above, the randomaccess procedure includes transmission of a random access preamble,reception of a random access response, and subsequent reception ofMessage 3 (Msg3). That is, the terminal device 10 transmits the randomaccess preamble to the base station 20 (gNB).

For example, according to 3GPP TS 38.211, as for the preamblestransmitted by terminal device 10, 64 different preambles are allocatedto each RACH occasion. Therefore, when a plurality of terminal devices10 transmit the preambles at the same RACH occasion, in a case wheredifferent preambles are used, the base station 20 can separate anddiscriminate the preambles. RACH-Config including the RACH occasion isnotified by system information (e.g., system information block (SIB) 1)provided by the base station 20. The SSB index corresponding to acertain beam and the RACH occasion can be associated with each other ina one-to-one relationship. Therefore, by confirming at which RACHoccasion the terminal device 10 has transmitted the preamble, the basestation 20 can identify the best SSB index, that is, the best beam forthe terminal device 10.

The base station 20 (gNB) transmits the random access response inresponse to the reception of the random access preamble. Since the RACHoccasion at which the random access preamble is transmitted isassociated with the best SSB index or the best beam for the terminaldevice 10, the base station 20 can recognize the best beam for theterminal device 10 based on the SSB index associated with the RACHoccasion. Then, the random access response transmitted from the basestation 20 is beamformed and transmitted in the same direction as thebeam corresponding to the SSB index. In subsequent communications (e.g.,transmission of Msg3 and RRC setup procedure), a beam directed in thesame direction is used unless the beam is switched.

The application server 30 determines to provide the application (e.g.,AR image data) to the terminal device 10 at an arbitrary timing. Thisdetermination may be performed on the basis of an explicit or implicitrequest from the terminal device 10. The explicit request here may bemade by transmitting a request message from the terminal device 10 tothe application server 30, or the implicit request may be determinationin the application server 30 under the condition that subscription dataof the terminal device 10 indicates that provision of the application issubscribed or allowed. Then, in Step S106, the application server 30acquires information (e.g., SSB index) regarding the best beam for theterminal device 10. For example, the application server 30 may acquirethe information regarding the SSB index from the base station 20 (gNB)via the core network node 40. For example, the application server 30 mayacquire the information regarding the SSB index via the API provided bythe base station 20 (gNB) or the core network node 40. In a case wherethe present embodiment is applied to the edge computing, the API isprovided to the application server 30 via a reference point “EDGE-7”between the 3GPP Network (including the base station 20 (gNB) and thecore network) and the edge application server. For example, the edgeapplication server can access a 3GPP network function and the API (viathe API exposed by the NEF). In addition or instead, the API may beprovided to the application server 30 via a reference point “EDGE-2”between the 3GPP Network (including the base station 20 (gNB) and thecore network) and an edge enabler server, and a reference point “EDGE-3”between the edge enabler server and the edge application server. Inaddition or instead, the API may be provided to the application server30 via a reference point “EDGE-8” between the 3GPP Network (includingthe base station 20 (gNB) and the core network) and an edge data networkconfiguration server, a reference point “EDGE-6” between the edge datanetwork configuration server and the edge enabler server, and thereference point “EDGE-3” between the edge enabler server and the edgeapplication server. Details thereof will be described later.

Then, the application server 30 determines the correction information(e.g., angle/distance) used for displaying the AR image data associatedwith the SSB index in advance based on the acquired informationregarding the SSB index. Then, in Step S107, the application server 30renders the AR image data by using the correction information, andtransmits the rendered (e.g., aligned using the correction information)corrected AR image data to the terminal device 10. Note that therendering here may be any one of the cloud rendering, the edgerendering, or the split rendering described above.

In Step S108, the terminal device 10 displays the received corrected ARimage data on the display 130. As a result, the virtual object includedin the corrected AR image data can be superimposed on the real objectimaged by the camera 140.

In this way, the corrected AR image data that has been appropriatelyaligned according to the location of each user (terminal device 10) canbe provided to each user (terminal device 10). The user can view the ARimage M1 (see FIG. 1) in which the corrected AR image data that has beenappropriately aligned and the real image are superimposed.

Note that in a case where the use case to which the present embodimentis applied is an AR/VR event in a large-scale facility such as thestadium ST, a possibility that (the user who uses) the terminal device10 moves is lower as compared with other cases. Therefore, even in acase where only one reference point imaged by the camera 140 included inthe terminal device 10 and the correction information associated withthe best beam (SSB) for the terminal device 10 (the direction anddistance of the area covered by one beam from the reference point) areused to align the virtual object, the AR image can be displayed on thedisplay to the extent that the user does not feel uncomfortable.

<1.5. Modified Examples>

<1.5.1. First Modified Example>

The first embodiment describes a case where the application server 30generates the corrected image data obtained by correcting the AR imagedata based on the correction information. In addition to the aboveexample, the application server 30 may transmit the correctioninformation associated with the best beam to the terminal device 10, andthe terminal device 10 may correct the AR image data. Therefore, in afirst modified example of the first embodiment, a case where theterminal device 10 generates the corrected AR image data based on thecorrection information will be described.

FIG. 12 is a sequence diagram illustrating an operation example of thecommunication system according to the first modified example of thefirst embodiment of the present disclosure. The operation of thecommunication system illustrated in FIG. 12 until the application server30 acquires the information regarding the best beam in Step S106 is thesame as the operation illustrated in FIG. 8. The application server 30that has acquired the information regarding the best beam in Step S106determines the correction information (e.g., angle/distance) used fordisplaying the AR image data associated with the information regardingthe SSB index based on the acquired information regarding SSB indexinformation.

The application server 30 determines to provide the application (e.g.,AR image data) to the terminal device 10 at an arbitrary timing. Thetiming of the determination is the same as the operation illustrated inFIG. 8.

Then, in Step S201, the application server 30 transmits the AR imagedata and the correction information corresponding to the best beam A tothe terminal device 10.

In Step S202, the terminal device 10 performs correction such asalignment on the received AR image data by using the correctioninformation.

Subsequently, in Step S108, the virtual object included in the correctedAR image data is superimposed on the real object (real image) imaged bythe camera 140 and displayed on the display 130.

In this way, the terminal device 10 can appropriately perform alignmentby using the correction information according to the location of eachuser (terminal device 10). The user can view the AR image M1 (seeFIG. 1) in which the corrected AR image data that has been appropriatelyaligned and the real image are superimposed.

<1.5.2. Second Modified Example>

In the above first modified example, the application server 30determines the correction information based on the best beam. Inaddition to the above example, the terminal device 10 may determine thecorrection information based on the best beam. Therefore, in a secondmodified example of the first embodiment, a case where the terminaldevice 10 determines the correction information based on the best beamwill be described.

FIG. 13 is a sequence diagram illustrating an operation example of thecommunication system according to the second modified example of thefirst embodiment of the present disclosure. The operation of thecommunication system illustrated in FIG. 13 until the application server30 acquires the information regarding the best beam in Step S106 is thesame as the operation illustrated in FIG. 8.

The application server 30 determines to provide the application (e.g.,AR image data) to the terminal device 10 at an arbitrary timing. Thetiming of the determination is the same as the operation illustrated inFIG. 8.

In Step S301, the application server 30 transmits the AR image data andthe correction information corresponding to all the beams to theterminal device 10. For example, the application server 30 transmits allcombinations of the beam (SSB index) and the correction information tothe terminal device 10.

In Step S302, the terminal device 10 selects the correction informationcorresponding to the best beam (here, beam A) determined in Step S104from a plurality of pieces of received correction information.

Note that the subsequent operations are the same as the operations ofthe communication system of the first modified example illustrated inFIG. 12.

In this way, the terminal device 10 can appropriately align the AR imagedata by selecting the correction information according to the locationof each user (terminal device 10). The user can view the AR image M1(see FIG. 1) in which the corrected AR image data that has beenappropriately aligned and the real image are superimposed.

2. Second Embodiment

In a second embodiment, details of generation of the AR image dataperformed by the application server 30 according to the first embodimentwill be described. Specifically, an example in which reduction of anunviewable part (information regarding a surface shape and a color onthe opposite side from the viewing direction) from virtual object data(AR image data) is made according to a location of a terminal device 10,and the virtual object data is provided to the terminal device 10 willbe described.

A configuration of a communication system in the present embodiment isthe same as the communication system illustrated in FIG. 4 in the firstembodiment. That is, the communication system in the present embodimentincludes the terminal device 10 (UE), a base station 20 (gNB), a corenetwork node 40 (e.g., UPF), and an application server 30 (e.g., (edge)application server).

In the present embodiment, data configured by a point cloud, which is aset of points having position information and attribute information (forexample, color information or reflection information) at the same timein a three-dimensional space, will be described as an example of thevirtual object data. However, specific examples of the virtual objectdata are not limited thereto. In other words, the virtual object datarendered as 3D data does not have to be the data configured by the pointcloud.

For example, in the point cloud, data is separated into geometry, whichindicates a three-dimensional structure, and attribute, which indicatescolor information or reflection information, and encoded. Octreeencoding as illustrated in FIG. 14 is used to compress geometry. Forexample, the octree encoding is a method of expressing the presence orabsence of points in each block by an octree in data expressed by avoxel. In this method, as illustrated in FIG. 14, a block with points isrepresented by 1 and a block without points is represented by 0. Notethat FIG. 14 is a diagram for describing a configuration of the pointcloud.

In a case where the point cloud is used for the AR image data, ageometry-based point cloud compression (G-PCC) stream in which 3Dstructure information of a point cloud object is uniformly compressed bythe octree encoding as illustrated in FIG. 14 is used for delivery. Notethat the term “G-PCC stream” may be an example of the virtual objectdata (AR image data) in the above-described first embodiment andmodified example. In this way, when uniformly compressed by the octreeencoding, the delivered G-PCC stream has three-dimensional informationthat is viewable from the surrounding 360°, and the fineness of theentire circumference is the same. In other words, whether or not thepoints included in the point cloud are dense (that is, whether or notthe delivered G-PCC stream has high definition) is proportional to theamount of data.

Therefore, in the present embodiment, when the G-PCC stream isgenerated, encoding is made by changing the fineness (octree depth) thatdivides the voxel for each part of the point cloud object, and changingthe definition for each part. For example, a portion (informationregarding the surface shape and color in the viewing direction) that isviewable from the terminal device 10 according to the location of theterminal device 10 is set to have high definition (depth=10) andencoded. On the other hand, a portion (information regarding the surfaceshape and color on the opposite side from the viewing direction) that isunviewable from the terminal device 10 according to the location of theterminal device 10 is set to have low definition (depth=5) or be notdrawn (depth=0) and encoded.

In the present embodiment, the viewable portion is determined(specified) based on the correction information (e.g., informationregarding the direction and distance of the area covered by one beam(SSB) from the reference point) described in the first embodiment. Theseprocessings may be performed in the application server 30. Then, theencoded AR image data (i.e., G-PCC stream) is provided from theapplication server 30 to the terminal device 10.

For example, information indicating a direction of the G-PCC streamrendered in high definition can be provided from the application server30 (media presentation description (MPD) file server) to the terminaldevice 10 (MPEG-DASH client) by extension of dynamic adaptive streamingover HTTP (DASH MPD). The media presentation description (MPD) isdescribed in XML and includes Presentation, Period, AdaptationSet,Representation, and Segment. Among these, AdaptationSet represents unitssuch as video, audio, and subtitles, and includes a plurality ofRepresentations. Representation is information such as a video/audio bitrate, a resolution, and an aspect ratio. Information (field) indicatingthe direction in which rendering is made in high definition can be newlydefined in this AdaptationSet. That is, the information (field) can benewly defined in an attribute “direction” of an element“gpcc:directionInfo”. Further, in a case where the correctioninformation (e.g., the information regarding the direction and distanceof the area covered by one beam (SSB) from the reference point)described in the first embodiment is signaled to the terminal device 10(e.g., a case where the sequence of FIG. 12 or the sequence of FIG. 13is adopted), these pieces of correction information may also be newlydefined as a field (i.e., the attribute “direction” of the element“gpcc:directionInfo”) in AdaptationSet.

In addition, six directions including 0: X+, 1: Y+, 2: X−, 3: Y−, 4: Z+,and 5: Z− can be set as possible values of the attribute “direction”based on local coordinates of the point cloud. Also for the correctioninformation (e.g., information regarding the direction and distance ofthe area covered by one beam (SSB) from the reference point) describedin the first embodiment, six directions including 0: X+, 1: Y+, 2: X−,3: Y−, 4: Z+, and 5: Z− may be set. A coordinate system of the localcoordinates of the point cloud and a coordinate system for indicatingthe direction and the distance included in the correction informationdescribed in the first embodiment may be matched (synchronized).Alternatively, instead, at least one coordinate axis (e.g., x axis) ofthe local coordinates of the point cloud may be matched (synchronized)with a direction indicated by L′ (the distance between a central portionof an area covered by a beam identified by a certain SSB index and thereference point) described in the first embodiment.

Here, a spatial division method and spatial position information will bedescribed.

For example, the shape of the point cloud object is changed frame byframe at the maximum. Therefore, spatial division is performed byapplying a certain division rule that does not depend on the change inthe shape of the point cloud object. Specifically, a partial point cloudobject contained in a rectangular parallelepiped block (hereinafter,appropriately referred to as a block) that occupies relatively the samespatial position with respect to a box containing the entire point cloudobject (hereinafter, appropriately referred to as an object box) isencoded as a single partial G-PCC stream.

FIG. 15 is a diagram for describing the spatial division method and thespatial position information according to the second embodiment of thepresent disclosure. FIG. 15 illustrates an example of dividing theobject box in half in an X-axis direction.

As illustrated in FIG. 15, the object box containing the entire pointcloud object at a time t0 is divided in half in the x-axis directioninto partial point cloud objects t0-a and t0-b. Similarly, at a time t1,the object box is divided into partial point cloud objects t1-a andt1-b, and at a time t2, the object box is divided into partial pointcloud objects t2-a and t2-b. Then, the G-PCC stream of a includes thepartial point cloud object t0-a, the partial point cloud object t1-a,and the partial point cloud object t2-a. On the other hand, the G-PCCstream of b includes the partial point cloud object t0-b, the partialpoint cloud object t1-b, and the partial point cloud object t2-b. Notethat, in FIG. 15, at an arbitrary time t, the entire point cloud objectis divided into the partial point cloud object a and the partial pointcloud object b in the x-axis direction, but the present invention is notlimited thereto. For example, the entire point cloud object may bedivided in a y-axis direction or may be divided in a z-axis directionaccording to the viewing direction of the user. Alternatively, instead,in a case where at least one coordinate axis (e.g., x axis) is notmatched (synchronized) with the direction indicated by L′ (the distancebetween a central portion of an area covered by a beam identified by acertain SSB index and the reference point) described in the firstembodiment, the division described in FIG. 15 may be division in thedirection indicated by L′ described in the first embodiment.

According to this method, a relative spatial position of the partialpoint cloud object contained in the partial G-PCC stream with respect tothe entire point cloud object is dynamically invariant. In a case wherethe relative spatial position is dynamically changed, a relationshipbetween a viewing portion and the partial G-PCC stream containing theviewing portion is dynamically changed. Therefore, when a clientacquires the G-PCC stream containing the viewing portion whosedefinition is enhanced, it becomes necessary to switch thehigh-definition G-PCC stream to be acquired even in a case where theviewing portion is invariant. Therefore, by this spatial divisionmethod, it is possible to eliminate the need to switch thehigh-definition G-PCC stream to be acquired when the viewing portion isinvariant.

[Operation Example of Application Server (MPD File Server)]

FIG. 16 is a flowchart for describing generation processing in which theapplication server 30 (MPD file server) generates a file storing thepartial G-PCC stream. Note that it is a detailed operation example ofStep S107 illustrated in the sequence of FIG. 8 of the first embodiment,Step S201 illustrated in the sequence of FIG. 12 of the modifiedexample, and Step S301 illustrated in the sequence of FIG. 13.

In Step S401, the application server 30 (MPD file server) divides thepoint cloud object and generates each partial point cloud object, and atthe same time, generates the spatial position information and groupinginformation. More specifically, the application server 30 (MPD fileserver) generates the partial point cloud object corresponding to aportion that is viewable from the terminal device 10, and the partialpoint cloud object corresponding to a portion that is unviewable fromthe terminal device 10 by using the correction information (informationregarding the direction and distance of the area that can be covered byone beam (SSB) from the reference point (real object)) associated withthe best beam (SSB index) for the terminal device 10 (MPEG-DASH client).More specifically, the entire point cloud object is divided into aplurality of objects in the direction of the area that can be covered byone beam (SSB) from the reference point (real object).

In Step S402, the application server 30 (MPD file server) sets theoctree depth of each partial point cloud object and then performs G-PCCencoding. As a result, the application server 30 (MPD file server)generates the partial G-PCC stream. At the same time, the applicationserver 30 (MPD file server) generates definition information. Morespecifically, the application server 30 (MPD file server) sets theoctree depth to a predetermined value (e.g., 10) for the partial pointcloud object corresponding to the portion that is viewable from theterminal device 10, and sets the octree depth to a smaller value (e.g.,5 or 0) for the partial point cloud object corresponding to the portionthat is unviewable from the terminal device 10.

In Step S403, the application server 30 (MPD file server) stores eachpartial G-PCC stream in an individual file and records the file in amemory.

In Step S404, the application server 30 (MPD file server) generates theMPD including the spatial position information, the groupinginformation, and the definition information of each partial G-PCC streamand stores the MPD in the memory. Then, the application server 30 (MPDfile server) provides the MPD to the terminal device 10 (MPEG-DASHclient) together with the file storing the partial G-PCC stream andrecorded in the memory.

[Operation Example of Terminal Device (MPEG-DASH Client)]

FIG. 17 is a flowchart for describing reproduction processing in whichthe terminal device 10 (MPEG-DASH client) reproduces the file storingthe partial G-PCC stream. Note that it is a detailed operation exampleof Step S107 (Step S201 illustrated in the sequence of FIG. 12 of themodified example and Steps S301 illustrated in the sequence of FIG. 13)and S108 illustrated in the sequence of FIG. 8 of the first embodiment.

In Step S501, the terminal device 10 (MPEG-DASH client) acquires theMPD. More specifically, the MPD is provided from the application server30 (MPD file server) to the terminal device 10 (MPEG-DASH client).

In Step S502, the terminal device 10 (MPEG-DASH client) identifiesAdaptation Set of a viewable partial G-PCC stream and Adaptation Set ofan unviewable part G-PCC based on the spatial position information ofthe MPD acquired in Step S501.

In Step S503, the terminal device 10 (MPEG-DASH client) selectshigh-definition Representation for the viewable partial G-PCC streambased on the definition information of the MPD.

In Step S504, the terminal device 10 (MPEG-DASH client) selectslow-definition Representation for the unviewable partial G-PCC streambased on the definition information of the MPD.

In Step S505, the terminal device 10 (MPEG-DASH client) acquires all thepartial G-PCC streams referenced from Representation selected in StepS503 and Step S504.

In Step S506, the terminal device 10 (MPEG-DASH client) decodes theacquired partial G-PCC stream, reconstructs the point cloud object basedon the spatial position information, and renders a display screen. Then,the rendered AR image is displayed on the display of the terminal device10 (MPEG-DASH client).

In Step S507, the terminal device 10 (MPEG-DASH client) determineswhether or not the end of the stream has been reached. In a case wherethe terminal device 10 (MPEG-DASH client) determines in Step S507 thatthe end of the stream has not been reached, the processing proceeds toStep S508.

In Step S508, the terminal device 10 (MPEG-DASH client) determineswhether or not a field-of-view direction (viewing direction) has beenchanged, and in a case where it is determined that the field-of-viewdirection has not been changed, the processing returns to Step S506. Ina case where it is determined that the field-of-view direction has beenchanged, the processing returns to Step S502, and the same processing isrepeated thereafter. The change in field-of-view direction in Step S508may be detected by various sensors (at least one of the sensorsdescribed above) provided in the terminal device 10 (MPEG-DASH client)or may be detected based on a change (beam switching) of the SSB indexcorresponding to the best beam for the terminal device 10 (MPEG-DASHclient).

On the other hand, in Step S507, in a case where the terminal device 10(MPEG-DASH client) determines that the end of the stream has beenreached, the processing ends.

As a result, the terminal device 10 acquires the G-PCC stream encoded soas to have high definition for the portion that is viewable from thelocation of the terminal device 10 (information regarding the surfaceshape and color in the viewing direction), and can acquire the G-PCCstream encoded so as to have low definition for other portions. As aresult, it is possible to align the point cloud object in considerationof the location of the terminal device 10 with respect to the realobject and output the AR image while suppressing the amount of data fromthe application server 30 to the terminal device 10. In particular, in alarge-scale facility such as a stadium, the number of terminal devices10 is expected to be enormous, and thus, when assuming the use case suchas an AR/VR event, the limitation of network bandwidth can become abottleneck. Therefore, suppressing the amount of data for one usercontributes to preventing deterioration of the quality of experience forthe user. Furthermore, in a case where the use case to which the presentembodiment is applied is an AR/VR event in a large-scale facility suchas the stadium ST, a possibility that (the user who uses) the terminaldevice 10 moves is lower as compared with other cases. This is becausethe user who views the AR/VR event views the AR/VR event while sittingin a seat in a stadium or the like. Therefore, as in the presentembodiment, even in a case where the portion that is unviewable from theterminal device 10 (the information regarding the surface shape andcolor on the opposite side from the viewing direction) is set to havelow definition (depth=5) or be not drawn (depth=0) and is encoded, thepossibility or frequency that the unviewable part become viewable bymovement of the terminal device 10 is low, and from this viewpoint aswell, it is possible to contribute to preventing deterioration of thequality of experience of the user.

3. Third Embodiment

In a third embodiment, application examples of the first and secondembodiments and modified examples will be described.

In the first and second embodiments and modified examples describedabove, acquisition of the information of the 3GPP network (including thebase station and the core network) acquired by the application server 30(e.g., the information regarding the above-described beam (e.g., the SSBindex and the antenna setting information list corresponding thereto)and the SSB index of the best beam for the terminal device 10 to whichthe AR image data is provided) using the API may be implemented by thearchitecture of the edge computing and various APIs used for the edgecomputing.

FIG. 18 is a diagram illustrating an example of an applicationarchitecture to which the edge computing is applied. The diagramillustrated in FIG. 18 is disclosed, for example, in 3GPP TR 23.758.

The terminal device 10 in the first and second embodiments and modifiedexamples described above may correspond to a UE of FIG. 18.Alternatively, instead, the terminal device 10 in the first and secondembodiments and modified examples described above may correspond to atleast one of an “application client(s)” or an “edge enabler client” inthe UE of FIG. 18. The base station 20 (e.g., gNB) and one or more corenetwork nodes 40 (e.g., UPF, AMF, SMF, and NEF) in the first and secondembodiments and modified examples described above may be included in a“3GPP network” of FIG. 18. The application server 30 in the first andsecond embodiments and modified examples described above may include atleast one of an “edge application server(s)” or an “edge enabler server”in an edge data network of FIG. 18. Alternatively, the “edge applicationserver(s)” and the “edge enabler server” in the edge data network may bedifferent application servers. In addition or instead, the applicationserver 30 in the first and second embodiments and modified examplesdescribed above may include an edge data network configuration server ofFIG. 18. The above-described “application function” may be included inthe “3GPP Network” (more specifically, the core network) of FIG. 18, ormay be included in the edge data network.

The edge enabler server provides a support function necessary for theedge application server to be operated on the edge data network. Thefunctions of the edge enabler server include:

provisioning of configuration information that enables the exchange ofan application data traffic with the edge application server, and

provision of information regarding the edge application server, such asavailability, to the edge enabler client.

Therefore, the edge enabler server may be referred to as a function (orlogical node) including at least a part of the above two functions.

The edge enabler client provides the support function necessary for theapplication client. The functions of the edge enabler client include:

acquisition and provisioning of configuration information that enablesthe exchange of the application data traffic with the edge applicationserver, and

detection of the edge application server available in the edge datanetwork.

Therefore, the edge enabler client may be referred to as a function (orlogical node) including at least a part of the above two functions.

The edge enabler server exposes a location reporting API to the edgeapplication server. The exposure is performed to support tracking andchecking of a valid location of the UE. For the location reporting APIexposed by the edge enabler server, an API (e.g., northbound API) can berelayed (relayed or forwarded) in the NEF to monitor the location of theUE (an event related to the location). The edge application server canrequest the location reporting API for one-time reporting (singlereporting) to check the location of the current UE. The edge applicationserver can also request the location reporting API for continuousreporting to track the location of the UE.

FIG. 19 is a sequence diagram illustrating an example of a processingprocedure of a communication system according to the third embodiment ofthe present disclosure. FIG. 19 illustrates an example of a procedurefor detection or acquisition of the location of the UE (terminal device10) by the edge enabler server from a 3GPP system (the base station 20,the network including the core network node 40, or the system in thefirst and second embodiments and modified examples described above).This sequence may be performed via a reference point “EDGE-2” asdescribed in FIG. 18. Note that the 3GPP system is also referred to asthe 3GPP network. Further, the sequence diagram of FIG. 19 may be atleast one detailed example of Step S101 and Step S106 of the sequenceillustrated in FIG. 8 in the first embodiment.

In Step S601, the edge enabler server interacts (e.g., communication)with the 3GPP system (e.g., 5GS or EPS) to acquire the location of theUE. For example, the edge enabler server can use the API exposed by theNEF. The edge enabler server can request the 3GPP system to performcontinuous location reporting for updating location information of theUE in order to avoid repeated requests for location reporting to the3GPP system. As a result, the edge enabler server can detect the latestlocation of the UE at any time. The location information of the UEprovided by the 3GPP system to the edge enabler server may include atleast one of GPS coordinates, a cell ID, a tracking area ID, orinformation indicating an address (street or district). In addition orinstead, the location information of the UE provided by the 3GPP systemto the edge enabler server may include the beam identifier (e.g., SSBindex) in the first and second embodiments and modified examplesdescribed above. An index of the CSI-RS or an index of a positioningreference signal may be included instead of the SSB index.

The edge enabler server can consider granularity of the locationinformation (e.g., GPS coordinates, a cell ID, a tracking area ID, anaddress, and a beam identifier (e.g., SSB index)) requested by the edgeapplication server.

Note that a detailed example of provision of the location information ofthe UE from the 3GPP system to the edge enabler server in Step S601 ofFIG. 19 will be described later.

FIG. 20 is a sequence diagram illustrating an example of a processingprocedure of the communication system according to the third embodimentof the present disclosure. FIG. 20 illustrates an example in which theedge application server acquires a UE (terminal device 10) locationreport from the edge enabler server via the location reporting APIdescribed above. This sequence may be performed via a reference point“EDGE-3” as described in FIG. 18. Further, the sequence diagram of FIG.20 may be at least one detailed example of Step S101 and Step S106 ofthe sequence illustrated in FIG. 8 in the first embodiment.

In Step S701, the edge application server transmits a location reportingAPI Request message to the edge enabler server to request the locationreporting API. This message includes information regarding theidentifier and location of the UE (terminal device 10) (e.g., positiongranularity). The position granularity indicates a format of thereported location information (e.g., at least one of GPS coordinates, acell ID, a tracking area ID, information indicating an address (streetor district), or a beam identifier (e.g., SSB index)).

In Step S702, the edge enabler server checks the location of the UE(terminal device 10).

In Step S703, the edge enabler server considers the granularity of therequested location and returns the location information (e.g., at leastone of GPS coordinates, a cell ID, a tracking area ID, and informationindicating an address (street or district), or a beam identifier (e.g.,SSB index)) of the UE (terminal device 10) as a response message(location reporting API response message). The response message mayinclude a time stamp of the location of the UE (terminal device 10).

Note that the location information of the UE (terminal device 10) doesnot have to be reported based on an explicit request from the edgeapplication server. For example, the edge application server maysubscribe to the location reporting API for the edge enabler server. Inthis case, the edge enabler server may report the location information(e.g., at least one of GPS coordinates, a cell ID, a tracking area ID,and information indicating an address (street or district), or a beamidentifier (e.g., SSB index)) of the UE (terminal device 10) to the edgeapplication server when the edge enabler server detect the locationinformation of the UE (terminal device 10).

Further, the operation described with respect to the sequence diagramillustrated in FIG. 19 and the operation described with respect to thesequence diagram illustrated in FIG. 20 may be at least partiallycombined with each other. The combination thereof may be a detailedexample of the operation of at least one of Step S101 or Step S106 ofthe sequence illustrated in FIG. 8 in the first embodiment describedabove.

Further, in the present embodiment, the location information of the UE(terminal device 10) is provided from the 3GPP system to the edgeapplication server via the reference point “EDGE-2” and the referencepoint “EDGE-3”, but the present invention is not limited thereto. Forexample, the provision of the location information of the UE (terminaldevice 10) from the 3GPP system to the edge application server may beperformed directly from the 3GPP system to the edge application servervia a reference point “EDGE-7”. Alternatively, instead, for example, theprovision of the location information of the UE (terminal device 10)from the 3GPP system to the edge application server may be performed viareference points “EDGE-8”, “EDGE-6”, and “EDGE-3”.

4. Fourth Embodiment

In a fourth embodiment, application examples of the first, second, andthird embodiments and modified examples will be described. Morespecifically, a detailed example of provision of the locationinformation of the UE from the 3GPP system to the Edge Enabler Server inStep S601 of FIG. 19 in the third embodiment will be described later.

In the present embodiment, an NR Positioning Protocol A (NRPPa)specified in 3GPP TS 38.455 may be used for the location information ofthe UE from the 3GPP system to the Edge Enabler Server. The NRPPadefines a protocol related to location information between an NG-RANnode (e.g., the base station 20 described above or below) and a LocationManagement Function (LMF), and provides at least the following twofunctions:

Enhanced Cell-ID (E-CID (positioning method)) Location InformationTransfer, and

Observed Time Difference of Arrival (OTDOA) Information Transfer.

That is, even in a case where the UE location information reported fromthe NG-RAN node to the LMF is provided to the Edge Enabler Server (orEdge Application server or Edge Data Network Configuration Server) viathe NEF or directly using the API. Note that the LMF may be the corenetwork node 40 (e.g., a node included in 5GS/EPS in FIG. 19) describedabove or to be described later.

The E-CID Location Information Transfer in the NRPPa allows the NG-RANnode to exchange the location information with the LMF for E-CIDpositioning. The E-CID Location Information Transfer includes thefollowing procedures:

a) E-CID Measurement Initiation,

b) E-CID Measurement Failure Indication,

c) E-CID Measurement Report, and

d) E-CID Measurement Termination.

The OTDOA Information Transfer in the NRPPa allows the NG-RAN node toexchange the location information with the LMF for OTDOA positioning.The OTDOA Information Transfer includes an OTDOA Information Exchangeprocedure.

FIG. 21 is a sequence diagram illustrating the E-CID MeasurementInitiation procedure.

In Step S801 of FIG. 21, the LMF transmits an E-CID MeasurementInitiation Request message to the NG-RAN node. The E-CID MeasurementInitiation Request message includes an Information Element (IE) “MessageType”, an IE “NRPPa Transaction ID”, an IE “LMF UE Measurement ID”, andan IE “Report Characteristics”, and may further include at least one ofan IE “Measurement Periodicity” or an IE “Measurement Quantities”(including at least one IE “Measurement Quantities Item”). The IE“Measurement Quantities Item” specifies the type of Measurement Quantityto be reported to the NG-RAN. In the IE “Measurement Quantities Item”,at least one of Cell-ID, an Angle of Arrival, Timing Advance Type 1,Timing Advance Type 2, RSRP, or RSRQ is set. In the present embodiment,in addition to or instead of these, a beam identifier (e.g., SSB Index)may be set in the IE “Measurement Quantities Item”. The SSB Index may bean identifier of the best beam for the terminal device 10 describedabove or below.

In a case where the NG-RAN node can initiate the requested E-CIDmeasurement, in Step S802, the NG-RAN node transmits an E-CIDMEASUREMENT INITIATION RESPONSE message to the LMF. The E-CIDMEASUREMENT INITIATION RESPONSE message includes an IE “Message Type”,an IE “NRPPa Transaction ID”, an IE “LMF UE Measurement ID”, and an IE“RAN UE Measurement ID”, and may further include at least one of an IE“E-CID Measurement Result” or an IE “Cell Portion ID”. The IE “E-CIDMeasurement Result” includes a Serving Cell ID (an NG-RAN Cell GlobalIdentifier of the serving cell) and a Serving Cell Tracking Area Code(TAC), and may further include an IE “NG-RAN Access Point Position”. TheIE “NG-RAN Access Point Position” is used to identify the geographiclocation of the NG-RAN node. The IE “NG-RAN Access Point Position” mayindicate, for example, location information for identifying locations ofa plurality of base stations 20 set in the stadium ST described above orbelow. The IE “Cell Portion ID” indicates the location (cell portion) ofa target UE (terminal device 10) in a cell. The current specificationsspecify that the Cell Portion ID can be set to any of the integers of 0,1, . . . , and 4095. The Cell Portion ID may correspond to theidentifier of the best beam (e.g., SSB Index) for the terminal device 10described above or below. That is, the value of the SSB Index (e.g., 0,1, . . . , or 64) and the value of Cell Portion ID may be matched or maybe associated with each other.

Note that, in a case where “OnDemand” is set in the IE “ReportCharacteristics” in the E-CID Measurement Initiation Request message,the NG-RAN node may include, in the E-CID MEASUREMENT INITIATIONRESPONSE message to be returned, at least one of the IE “E-CIDMeasurement Result” or the IE “Cell Portion ID”.

In a case where “Periodic” is set in the IE “Report Characteristics” inthe E-CID Measurement Initiation Request message, the NG-RAN nodereports, to the LMF, at least one of the IE “E-CID Measurement Result”or the IE “Cell Portion ID” described above by using the E-CIDMeasurement Report procedure.

FIG. 22 is a sequence diagram illustrating the E-CID Measurement Reportprocedure.

In Step S901 of FIG. 22, the NG-RAN node (e.g., base station 20)transmits an E-CID MEASUREMENT REPORT message to the LMF. The E-CIDMEASUREMENT REPORT message may include at least one IE that is the sameas the IE contained in the E-CID MEASUREMENT INITIATION RESPONSE messagedescribed above.

In a case where the location information of the UE (e.g., terminaldevice 10) provided from the NG-RAN node (e.g., base station 20) to theLMF (e.g., core network node 40) is OTDOA information, the OTDOAInformation Exchange procedure is used as described above.

FIG. 23 is a sequence diagram illustrating the OTDOA InformationExchange procedure.

In Step S1001 of FIG. 23, the LMF transmits an OTDOA INFORMATION REQUESTmessage to the NG-RAN node. In Step S1002, in response to reception ofthis message, the NG-RAN node transmits an OTDOA INFORMATION RESPONSEmessage to the LMF. The OTDOA INFORMATION RESPONSE message contains atleast one of an IE “Message Type”, an IE “NRPPa Transaction ID”, or anIE “OTDOA Cells”. The IE “OTDOA Cells” indicate a Served cell(s) or aServed transmission point(s) that broadcasts a Positioning ReferenceSignal (PRS). The Served cell(s) refers to one or more cells served bythe NG-RAN node (e.g., base station 20). The Served transmissionpoint(s) indicates one or more transmission points (e.g., antennas)provided in the NG-RAN node (e.g., base station 20) (e.g., the gNB-DUdescribed above or below). The IE “OTDOA Cells” includes one or morepieces of OTDOA Cell Information. The OTDOA Cell Information may includeat least one of a cell ID (e.g., NG-RAN GlobalCell Identifier),frequency information, bandwidth information, an IE “NG-RAN Access PointPosition”, a PRS ID, or a transmission point (TP) ID. In addition, theOTDOA Cell Information may include a beam identifier (e.g., SSB Index)described above or described above. In addition or instead, the PRS IDincluded in the OTDOA Cell Information may be matched or be associatedwith the beam identifier (e.g., SSB Index) described above or describedabove. Note that the beam identifier described above or described abovemay be the identifier of the best beam for the terminal device 10described above or below.

By these procedures described in the present embodiment, the identifierof the best beam for the terminal device 10 is provided from the basestation 20 to the core network node 40. Therefore, the applicationserver 30 described above or below can acquire the SSB Index of the bestbeam for the terminal device 10 to which the AR image data is provided,from the core network 40 via the API or the like (for example, using theprocedure of the third embodiment).

The procedure for providing the UE location information from the NG-RANnode to the LMF in the present embodiment may be performed independentlyof other embodiments (e.g., the first, second, and third embodiments andmodified examples) or may be performed in combination with otherembodiments.

5. Other Embodiments

The existing self-position estimation method may be applied to thevirtual object alignment in the first and second embodiments andmodified examples described above.

As a specific example of self-position estimation, in the terminaldevice 10 (e.g., AR device), the imaging unit 140 such as a cameraprovided in the terminal device 10 captures an image of a marker or thelike whose size is known on a real object in a real space. Then, theterminal device 10 analyzes the captured image to estimate at least oneof a relative position and posture of the terminal device 10 withrespect to the marker (or the real object on which the marker ispresented). Note that, in the following description, a case where theterminal device 10 estimates the position and posture thereof will bedescribed, but the terminal device 10 may estimate only one of theposition and the posture.

Specifically, it is possible to estimate a relative direction of theimaging unit 140 (or the terminal device 10 including the imaging unit140) with respect to the marker according to a direction of the markerin the image (for example, a direction of the shape or the like of themarker). In a case where the size of the marker is known, a distancebetween the marker and the imaging unit 140 (that is, the terminaldevice 10 including the imaging unit 140) can be estimated according tothe size of the marker in the image. More specifically, when the markeris imaged from a greater distance, the marker is imaged in a smallersize. Further, a range in the real space in the image at this time canbe estimated based on an angle of view of the imaging unit 140. Byutilizing the above characteristics, the distance between the marker andthe imaging unit 140 can be calculated back according to the size of themarker in the image (in other words, a proportion of the marker in theangle of view). With the above configuration, the terminal device 10 canestimate the relative position and posture thereof with respect to themarker.

Further, for example, the terminal device 10 according to the first andsecond embodiments and modified examples described above may be providedwith an acceleration sensor and an angular velocity sensor (gyrosensor), and may be configured to be able to detect the motion of thehead of the user who wears the terminal device 10 (in other words, themotion of the terminal device 10 itself). As a specific example, theterminal device 10 may detect components in a yaw direction, a pitchdirection, and a roll direction as the motion of the head of the user,thereby recognizing a change of at least one of the position and theposture of the head of the user.

Further, a technology called simultaneous localization and mapping(SLAM) may be used for the self-position estimation of the terminaldevice 10. The SLAM is a technology of performing self-locationestimation and environment map creation in parallel by using the imagingunit 140 such as a camera, various sensors, and an encoder. As a morespecific example, in the SLAM (particularly, visual SLAM), athree-dimensional shape of an imaged scene (or subject) is sequentiallyrestored based on a moving image captured by the imaging unit 140. Then,by associating the restoration result of the imaged scene with a resultof detecting the position and posture of the imaging unit 140, a map ofthe surrounding environment is created and the position and posture ofthe imaging unit 140 (or the terminal device 10) in the environment areestimated. Note that the position and posture of the imaging unit 140can be estimated as information indicating a relative change based onthe detection result of the sensor by, for example, providing varioussensors such as an acceleration sensor and an angular velocity sensor inthe terminal device 10. It is a matter of course that in a case wherethe position and posture of the imaging unit 140 can be estimated, themethod is not necessarily limited to the method based on the detectionresults of various sensors such as an acceleration sensor and an angularvelocity sensor.

With the above configuration, for example, a result of estimating therelative position and posture of the terminal device 10 with respect tothe marker based on the result of imaging the known marker by theimaging unit 140 may be used for initialization processing or positioncorrection in the SLAM described above. With such a configuration, theterminal device 10 can estimate the position and posture thereof withrespect to the marker (or the real object on which the marker ispresented) by performing self-position estimation based on the SLAM thathas received a result of previously performed initialization or positioncorrection even in a situation where the marker is not within the angleof view of the imaging unit 140.

The above method may be used together with the alignment method in thefirst and second embodiments and modified examples described above. Forexample, the alignment method in the first and second embodiments andmodified examples described above may be used for the initializationprocessing and position correction in the SLAM. Highly accuratealignment of the virtual object with respect to the real object can beimplemented by a plurality of combinations of the first and secondembodiments and modified examples described above and the known methods.

In addition to the above-described stadium ST, examples of thelarge-scale facility may include the following facilities. For example,examples of the large-scale facility may include a concert hall, atheater, a live house, a plaza, a stadium, a circuit, a racetrack, abicycle racetrack, a skate link, a movie theater, and an arena.

Examples of the synchronization signal include the SSB, the CSI-RS, thepositioning reference signal, and the like. That is, in some of theembodiments and modified examples described above, the CSI-RS or thepositioning reference signal may be used instead of the SSB. The SSBIndex in some of the embodiments described above may be replaced with aCSI-RS identifier (e.g., CSI-RS resource indicator (CRI)) or a PRSidentifier (PRS-ID).

In addition, the first and second embodiments and modified examplesdescribed above have been described mainly for 3GPP 5G NR Standalone,but the application is not limited thereto. For example, the first andsecond embodiments and modified examples described above may be appliedmainly to 3GPP 5G NR Non-Standalone.

As described above, a cell provided by the base station 20 is called aServing cell. The Serving cell includes a primary cell (PCell) and asecondary cell (SCell). In a case where Dual Connectivity (e.g.EUTRA-SUTRA Dual Connectivity, EUTRA-NR Dual Connectivity (ENDC),EUTRA-NR Dual Connectivity with SGC, NR-EUTRA Dual Connectivity (NEDC),or NR-NR Dual Connectivity) is provided to the UE (e.g., terminal device10), the PCell and zero or one or more SCells provided by a master node(MN) are referred to as a master cell group. Further, the Serving cellmay include a primary secondary cell or a primary SCG Cell (PSCell).That is, in a case where the Dual Connectivity is provided to the UE,the PSCell and zero or one or more SCells provided by a secondary node(SN) are referred to as a secondary cell group (SCG). Unless speciallyconfigured (e.g., physical uplink control channel (PUCCH) on SCell), thePUCCH is transmitted by the PCell and the PSCell, not by the SCell.Radio link failure is detected in the PCell and the PSCell, and is notdetected (does not have to be detected) in the SCell. Since the PCelland the PSCell have a special role in the Serving cell(s) as describedabove, they are also called special cells (SpCells). One downlinkcomponent carrier and one uplink component carrier may be associatedwith one cell. Further, a system bandwidth corresponding to one cell maybe divided into a plurality of bandwidth parts. In this case, one ormore bandwidth parts may be set in the UE and one bandwidth part may beused in the UE as an active BWP. Further, radio resources (for example,a frequency band, numerology (subcarrier spacing), and slotconfiguration) that can be used by the terminal device 10 may bedifferent for each cell, each component carrier, or each BWP.

That is, the base station 20 in the first and second embodiments andmodified examples described above may be the MN or SN of the NR-NR DC asthe 3GPP 5G NR Standalone, or may be the gNB (en-gNB) in the ENDC, theENDC with 5GC, or the NEDC as the 3GPP 5G NR Non-Standalone.

Furthermore, local 5G may be applied to the communication systems insome of the embodiments and modified examples described above. Forexample, the base station (gNB) 20 (e.g., a plurality of gNBs arrangedin the stadium ST), the core network node (UPF) 40, and the applicationserver 30 in FIG. 4 may be operated as network nodes constituting thelocal 5G. For example, the stadium ST may be a local 5G service area.More specifically, a public land mobile network (PLMN) to which aplurality of gNBs arranged in the stadium ST and the UPF connected tothe gNB belong may be different from a PLMN of a mobile network providedby a mobile network operator (MNO), other than the stadium ST. In thiscase, the location information (i.e., beam identifier) of the terminaldevice 10 provided to the application server 30 via the base station 20and the core network node 40 may be provided to the application server30 via the base station 20 and the core network node 40 together withinformation indicating the local 5G network (e.g., an identifier of thelocal 5G, an ID of the PLMN that provides the local 5G, an identifier(global ID) of the base station 20 (e.g., gNB) belonging to the PLMNthat provides the local 5G, and an identifier (global ID) of the corenetwork node 40). The provision method may use the procedures, messages,and protocols in some of the embodiments described above. That is, theinformation indicating the local 5G network described above may beincluded in the message in some of the embodiments described above.

6. Supplementary Description

As described above, the preferred embodiments of the present disclosurehave been described in detail with reference to the accompanyingdrawings, but the technical scope of the present disclosure is notlimited to such examples. It will be apparent to those skilled in theart to which the present disclosure pertains that various modificationsor alterations can be conceived within the scope of the technical ideadescribed in the claims and it is naturally understood that thesemodifications or alterations fall within the technical scope of thepresent disclosure.

Furthermore, the effects described in the present specification aremerely illustrative or exemplary and are not restrictive. That is, thetechnology according to the present disclosure can exhibit, in additionto or in place of the above-described effects, other effects obvious tothose skilled in the art from the description of the presentspecification.

Note that the present technology can also have the followingconfigurations.

-   (1)

A terminal device comprising:

a transceiver;

a camera for imaging a real object;

a display for displaying an augmented reality image in which a virtualobject is superimposed on the real object imaged by the camera; and

a processor,

wherein the processor is configured to

receive at least one of a plurality of synchronization signalsbeamformed in directions different from each other and transmitted froma base station via the transceiver,

determine a first synchronization signal whose radio quality satisfies apredetermined threshold from the at least one received synchronizationsignal,

transmit a random access preamble by using a random access occasioncorresponding to the first synchronization signal in order to report thefirst synchronization signal to the base station, and

receive information regarding the augmented reality image from anapplication server after a random access processing procedure includingthe transmission of the random access preamble is completed,

the information regarding the augmented reality image is

correction information used for displaying the augmented reality image,or

augmented reality image data in which the virtual object is aligned withrespect to the real object based on the correction information,

in a case where the information regarding the augmented reality image isthe correction information, the processor aligns the virtual object withrespect to the real object by using the correction information,generates the augmented reality image, and outputs the augmented realityimage to the display,

in a case where the information regarding the augmented reality image isthe augmented reality image data in which the virtual object is alignedwith respect to the real object based on the correction information, theprocessor outputs the augmented reality image to the display based onthe received augmented reality image data, and

the correction information

is information for indicating a position of an area covered by thebeamformed and transmitted first synchronization signal with respect tothe real object, and

includes information regarding a direction of the virtual object to bedisplayed on the display in the area and a distance from the real objectto the area.

-   (2)

The terminal device according to (1), wherein the correction informationis associated with an index of the beamformed and transmitted firstsynchronization signal.

-   (3)

The terminal device according to (1) or (2),

wherein the virtual object is a point cloud object,

the point cloud object includes a plurality of partial point cloudobjects, and

the plurality of partial point cloud objects have definition accordingto a view of a user.

-   (4)

The terminal device according to (3),

wherein the plurality of partial point cloud objects include a firstpartial point cloud object that is viewable by the user, and

an octree depth of the first partial point cloud object is set to havehigher definition than the other partial point cloud objects.

-   (5)

An application server comprising:

a network interface; and

a processor that generates an augmented reality image in which a virtualobject is superimposed on a real object imaged by a camera mounted on aterminal device,

wherein the processor is configured to

acquire, via a base station, information on a first synchronizationsignal determined by the terminal device from at least one of aplurality of synchronization signals beamformed in directions differentfrom each other and transmitted from the base station, and

transmit information regarding the augmented reality image to bedisplayed on a display mounted on the terminal device to the terminaldevice via the base station,

the information regarding the augmented reality image is

correction information to be used for displaying the augmented realityimage associated in advance with the first synchronization signal, or

augmented reality image data in which the virtual object is aligned withrespect to the real object based on the correction information,

in a case where the information regarding the augmented reality image isthe correction information, the augmented reality image data and thecorrection information are transmitted to the terminal device to causethe terminal device to align the virtual object with respect to the realobject by using the correction information,

in a case where the information regarding the augmented reality image isthe augmented reality image data, the processor aligns the virtualobject with respect to the real object by using the correctioninformation, generates the augmented reality image, and transmits theaugmented reality image to the terminal device, and

the correction information

is information for indicating a position of an area covered by thebeamformed and transmitted first synchronization signal with respect tothe real object, and

includes information regarding a direction of the virtual object to bedisplayed on the display in the area and a distance from the real objectto the area.

-   (6)

The application server according to (5), wherein the correctioninformation is associated with an index of the beamformed andtransmitted first synchronization signal.

-   (7)

The application server according to (5) or (6),

wherein the virtual object is a point cloud object,

the point cloud object includes a plurality of partial point cloudobjects, and

the plurality of partial point cloud objects have definition accordingto a view of a user.

-   (8)

The application server according to (7),

wherein the plurality of partial point cloud objects include a firstpartial point cloud object that is viewable by the user, and

an octree depth of the first partial point cloud object is set to havehigher definition than the other partial point cloud objects.

-   (9)

A receiving method for displaying an augmented reality image on aterminal device including

a transceiver,

a camera for imaging a real object,

a display for displaying the augmented reality image in which a virtualobject is superimposed on the real object imaged by the camera, and

a processor, the receiving method comprising:

receiving at least one of a plurality of synchronization signalsbeamformed in directions different from each other and transmitted froma base station via the transceiver;

determining a first synchronization signal whose radio quality satisfiesa predetermined threshold from the at least one received synchronizationsignal;

transmitting a random access preamble by using a random access occasioncorresponding to the first synchronization signal in order to report thefirst synchronization signal to the base station; and

receiving information regarding the augmented reality image from anapplication server after a random access processing procedure includingthe transmission of the random access preamble is completed,

wherein the information regarding the augmented reality image is

correction information to be used for displaying the augmented realityimage, or

augmented reality image data in which the virtual object is aligned withrespect to the real object based on the correction information,

in a case where the information regarding the augmented reality image isthe correction information, the virtual object is aligned with respectto the real object by using the correction information, the augmentedreality image is generated, and the augmented reality image is output tothe display,

in a case where the information regarding the augmented reality image isthe augmented reality image data in which the virtual object is alignedwith respect to the real object based on the correction information, theprocessor outputs the augmented reality image to the display based onthe received augmented reality image data, and

the correction information

is information for indicating a position of an area covered by thebeamformed and transmitted first synchronization signal with respect tothe real object, and

includes information regarding a direction of the virtual object to bedisplayed on the display in the area and a distance from the real objectto the area.

-   (10)

A transmitting method for transmitting, by an application serverincluding

a network interface and

a processor that generates an augmented reality image in which a virtualobject is superimposed on a real object imaged by a camera mounted on aterminal device,

information regarding the augmented reality image, the transmittingmethod comprising:

acquiring, via a base station, information on a first synchronizationsignal determined by the terminal device from at least one of aplurality of synchronization signals beamformed in directions differentfrom each other and transmitted from the base station; and

transmitting the information regarding the augmented reality image to bedisplayed on a display mounted on the terminal device to the terminaldevice via the base station,

wherein the information regarding the augmented reality image is

correction information to be used for displaying the augmented realityimage associated in advance with the first synchronization signal, or

augmented reality image data in which the virtual object is aligned withrespect to the real object based on the correction information,

in a case where the information regarding the augmented reality image isthe correction information, the augmented reality image data and thecorrection information are transmitted to the terminal device to causethe terminal device to align the virtual object with respect to the realobject by using the correction information,

in a case where the information regarding the augmented reality image isthe augmented reality image data, the virtual object is aligned withrespect to the real object by using the correction information, theaugmented reality image is generated, and the augmented reality image istransmitted to the terminal device, and

the correction information

is information for indicating a position of an area covered by thebeamformed and transmitted first synchronization signal with respect tothe real object, and

includes information regarding a direction of the virtual object to bedisplayed on the display in the area and a distance from the real objectto the area.

REFERENCE SIGNS LIST

10 UE

20 BASE STATION

30 APPLICATION SERVER

40 CORE NETWORK NODE

100, 200 ANTENNA UNIT

110, 210 COMMUNICATION UNIT (TRANSCEIVER)

120, 230 STORAGE UNIT (MEMORY)

130 DISPLAY UNIT (DISPLAY)

140 IMAGING UNIT (CAMERA)

150, 240, 330 CONTROL UNIT (PROCESSOR)

220, 310 NETWORK COMMUNICATION UNIT (NW INTERFACE)

1.-10. (canceled)
 11. A terminal device comprising: a transceiver; acamera for imaging a real object; a display for displaying an augmentedreality image in which a virtual object is superimposed on the realobject imaged by the camera; and a processor, wherein the processor isconfigured to receive, via the transceiver, at least one of a pluralityof synchronization signals beamformed in directions different from eachother and transmitted from a base station, determine a firstsynchronization signal whose radio quality satisfies a predeterminedthreshold from the at least one of the received synchronization signals,report the first synchronization signal to the base station, and receiveinformation regarding the augmented reality image from an applicationserver after the reporting to the base station, wherein the informationregarding the augmented reality image is correction information to beused for displaying the augmented reality image, or augmented realityimage data in which the virtual object is aligned with respect to thereal object based on the correction information, in a case where theinformation regarding the augmented reality image is the correctioninformation, the processor aligns the virtual object with respect to thereal object by using the correction information, generates the augmentedreality image, and outputs the augmented reality image to the display,in a case where the information regarding the augmented reality image isthe augmented reality image data in which the virtual object is alignedwith respect to the real object based on the correction information, theprocessor outputs the augmented reality image to the display based onthe received augmented reality image data, and the correctioninformation is information for indicating a position of an area, coveredby the beamformed and transmitted first synchronization signal, withrespect to the real object, and includes information regarding adirection of the virtual object to be displayed on the display in thearea and a distance from the real object to the area.
 12. The terminaldevice according to claim 11, wherein the reporting of the firstsynchronization signal to the base station is performed by transmittinga random access preamble using a random access occasion corresponding tothe first synchronization signal, and the information regarding theaugmented reality image is received from the application server after arandom access processing procedure including the transmission of therandom access preamble is completed.
 13. The terminal device accordingto claim 11, wherein the correction information is associated with anindex of the beamformed and transmitted first synchronization signal.14. The terminal device according to claim 11, wherein the virtualobject is a point cloud object, the point cloud object includes aplurality of partial point cloud objects, and the plurality of partialpoint cloud objects have definition according to a view of a user. 15.The terminal device according to claim 14, wherein the plurality ofpartial point cloud objects include a first partial point cloud objectthat is viewable by the user, and an octree depth of the first partialpoint cloud object is set to have higher definition than the otherpartial point cloud objects.
 16. An application server comprising: anetwork interface; and a processor that generates an augmented realityimage in which a virtual object is superimposed on a real object imagedby a camera mounted on a terminal device, wherein the processor isconfigured to acquire, via a base station, information of a firstsynchronization signal determined by the terminal device from at leastone of a plurality of synchronization signals beamformed in directionsdifferent from each other and transmitted from the base station, andtransmit information regarding the augmented reality image to bedisplayed on a display mounted on the terminal device to the terminaldevice via the base station, the information regarding the augmentedreality image is correction information to be used for displaying theaugmented reality image associated in advance with the firstsynchronization signal, or augmented reality image data in which thevirtual object is aligned with respect to the real object based on thecorrection information, in a case where the information regarding theaugmented reality image is the correction information, the augmentedreality image data and the correction information are transmitted to theterminal device to cause the terminal device to align the virtual objectwith respect to the real object by using the correction information, ina case where the information regarding the augmented reality image isthe augmented reality image data, the processor aligns the virtualobject with respect to the real object by using the correctioninformation, generates the augmented reality image, and transmits theaugmented reality image to the terminal device, and the correctioninformation is information for indicating a position of an area, coveredby the beamformed and transmitted first synchronization signal, withrespect to the real object, and includes information regarding adirection of the virtual object to be displayed on the display in thearea and a distance from the real object to the area.
 17. Theapplication server according to claim 16, wherein the correctioninformation is associated with an index of the beamformed andtransmitted first synchronization signal.
 18. The application serveraccording to claim 16, wherein the virtual object is a point cloudobject, the point cloud object includes a plurality of partial pointcloud objects, and the plurality of partial point cloud objects havedefinition according to a view of a user.
 19. The application serveraccording to claim 18, wherein the plurality of partial point cloudobjects include a first partial point cloud object that is viewable bythe user, and an octree depth of the first partial point cloud object isset to have higher definition than the other partial point cloudobjects.
 20. A terminal device control method for displaying anaugmented reality image on a terminal device including a transceiver, acamera for imaging a real object, a display for displaying the augmentedreality image in which a virtual object is superimposed on the realobject imaged by the camera, and a processor, the terminal devicecontrol method comprising: receiving, via the transceiver, at least oneof a plurality of synchronization signals beamformed in directionsdifferent from each other and transmitted from a base station;determining a first synchronization signal whose radio quality satisfiesa predetermined threshold from the at least one of the receivedsynchronization signals; reporting the first synchronization signal tothe base station; and receiving information regarding the augmentedreality image from an application server after the reporting to the basestation, wherein the information regarding the augmented reality imageis correction information to be used for displaying the augmentedreality image, or augmented reality image data in which the virtualobject is aligned with respect to the real object based on thecorrection information, in a case where the information regarding theaugmented reality image is the correction information, the virtualobject is aligned with respect to the real object by using thecorrection information, the augmented reality image is generated, andthe augmented reality image is output to the display, in a case wherethe information regarding the augmented reality image is the augmentedreality image data in which the virtual object is aligned with respectto the real object based on the correction information, the processoroutputs the augmented reality image to the display based on the receivedaugmented reality image data, and the correction information isinformation for indicating a position of an area, covered by thebeamformed and transmitted first synchronization signal, with respect tothe real object, and includes information regarding a direction of thevirtual object to be displayed on the display in the area and a distancefrom the real object to the area.
 21. The terminal device control methodaccording to claim 20, wherein the reporting of the firstsynchronization signal to the base station is performed by transmittinga random access preamble using a random access occasion corresponding tothe first synchronization signal, and the information regarding theaugmented reality image is received from the application server after arandom access processing procedure including the transmission of therandom access preamble is completed.
 22. A communication control methodfor transmitting, by an application server including a network interfaceand a processor that generates an augmented reality image in which avirtual object is superimposed on a real object imaged by a cameramounted on a terminal device, information regarding the augmentedreality image, the communication control method comprising: acquiring,via a base station, information of a first synchronization signaldetermined by the terminal device from at least one of a plurality ofsynchronization signals beamformed in directions different from eachother and transmitted from the base station; and transmitting theinformation regarding the augmented reality image to be displayed on adisplay mounted on the terminal device to the terminal device via thebase station, wherein the information regarding the augmented realityimage is correction information to be used for displaying the augmentedreality image associated in advance with the first synchronizationsignal, or augmented reality image data in which the virtual object isaligned with respect to the real object based on the correctioninformation, in a case where the information regarding the augmentedreality image is the correction information, the augmented reality imagedata and the correction information are transmitted to the terminaldevice to cause the terminal device to align the virtual object withrespect to the real object by using the correction information, in acase where the information regarding the augmented reality image is theaugmented reality image data, the virtual object is aligned with respectto the real object by using the correction information, the augmentedreality image is generated, and the augmented reality image istransmitted to the terminal device, and the correction information isinformation for indicating a position of an area, covered by thebeamformed and transmitted first synchronization signal, with respect tothe real object, and includes information regarding a direction of thevirtual object to be displayed on the display in the area and a distancefrom the real object to the area.